A Method for Planning Unmanned Aerial Vehicle (UAV) Inspection Tasks for Power Lines
By detecting uncaptured areas and adjusting the location of shooting points, the optimal inspection route is generated, solving the occlusion problem caused by changes in the location of the diversion line and improving the accuracy and efficiency of drone inspections of power lines.
Patent Information
- Application Number
- CN202611410322.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-11
- Publication Date
- 2026-11-17
AI Technical Summary
Existing drone inspection methods for power lines fail to effectively determine the occlusion status of current and subsequent shooting points when the location of the diversion line changes, leading to repeated shooting and increased flight distance. Furthermore, the flight costs for different shooting routes vary, affecting the accuracy and efficiency of the inspection task.
By detecting uncaptured areas and determining the range of changes in the location of the diversion line, the positions of the current and subsequent capture points are adjusted, a revised inspection route is generated, the coverage and additional flight distance of the two routes are compared, and the optimal updated inspection route is selected to ensure that unnecessary detours and repeated flights are reduced while meeting the capture requirements.
It improves the reliability of power line inspection tasks and the adaptability of route planning, reduces subsequent occlusion problems caused by individually correcting the current shooting task point, reduces the risk of missed shots caused by dynamic occlusion, and optimizes the economy of flight path.
Smart Images

Figure CN123113979A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of unmanned aerial vehicle (UAV) power line inspection technology, and more specifically, to a method for planning UAV inspection tasks for power lines. Background Technology
[0002] Existing methods for planning UAV inspections of power lines typically pre-define candidate shooting points that meet requirements for shooting distance, shooting angle, and image resolution based on the spatial location of towers, conductors, and components to be inspected. Combining UAV endurance constraints, the number of shooting points visited, and the flight distance between shooting points, a path optimization algorithm is used to determine the shooting points and their visiting order, generating an initial inspection route. The UAV executes shooting tasks sequentially according to the initial inspection route, acquiring images of the components to be inspected through its camera. The completion status of the inspection task is judged based on image quality, target recognition results, or preset shooting requirements. When the captured images are obstructed or do not meet shooting requirements, existing technologies can obtain supplementary images by adjusting the UAV position, changing the camera attitude, or adding supplementary shooting points, and make local adjustments to the inspection route to complete the image acquisition task for the components to be inspected.
[0003] The existing technology has the following shortcomings: During inspections, the guide line may shift position due to external factors such as wind, obstructing the observation path of the current shooting point and resulting in unshot areas of the inspected components. Since changes in the guide line's position may also affect subsequent shooting points, if only offset shooting is performed on currently unshot areas, or if existing subsequent shooting points are used for reshoots without a unified assessment and positional correction of obstruction at both current and subsequent shooting points, the subsequent routes may still contain shooting points that cannot be effectively captured. Furthermore, calculating flight distances for different reshoot routes based on either the original position or their adjusted positions can lead to inconsistent benchmarks for the increased flight distance, causing route selection to fail to accurately reflect the actual reshoot and subsequent flight costs. When the guide line continues to shift, already determined reshoot positions may be obstructed again, resulting in repeated reshoots, increased flight distances, or some inspected components failing to be effectively shot. Therefore, there is a need for a drone inspection task planning method that can combine the range of changes in the location of the diversion line to uniformly correct the current and subsequent shooting task points, compare the reshoot routes under the same remaining task conditions, and continuously update the method based on the actual shooting results.
[0004] To address the above problems, this invention proposes a solution. Summary of the Invention
[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method for planning unmanned aerial vehicle (UAV) inspection tasks for power lines, in order to solve the problems mentioned in the background art.
[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for planning unmanned aerial vehicle (UAV) inspection tasks for power lines, including the following steps; Select shooting task points that meet the preset number of visits, determine the visiting order based on the flight distance, and generate an initial inspection route; detect whether there are unshot areas on the parts to be inspected based on the images captured by the current shooting task points, and determine the range of position changes of the diversion line if there are unshot areas. Based on the range of changes in the location of the un-photographed area and the diversion line, determine whether the observation path of the current and subsequent photographing task points is blocked by the diversion line. Adjust the position of the affected photographing task points to obtain the corrected photographing position, and form the corrected subsequent inspection route according to the original visit order. Based on the corrected shooting position, a first route is generated after the current shooting task point is offset and reshot, and a second route is generated to supplement the unshot areas and rearrange the unfinished tasks using the corrected subsequent shooting task points. The coverage of the unshot areas by the two routes and the increase in flight distance relative to the corrected subsequent inspection route are compared to determine the updated inspection route. The inspection route is updated, and the location changes of un-photographed areas and diversion lines are updated. If un-photographed areas still exist, the location adjustment and route determination are repeated.
[0007] In a preferred embodiment, selecting shooting task points that meet the preset number of visits includes: forming a set task from multiple candidate shooting task points corresponding to the same component to be inspected; selecting shooting task points that meet the preset number of visits from the set task; determining the access order based on the flight distance between shooting task points; and generating an initial inspection route under the condition of meeting the drone's endurance constraints. Save the original shooting area and shooting requirements corresponding to each shooting task point, and record the areas that have been effectively shot and the incomplete status respectively. Do not assume that the corresponding shooting requirements have been completed simply because the shooting task point has been visited.
[0008] In a preferred embodiment, detecting uncaptured areas includes: registering the captured image with the original capture area of the component to be inspected; determining the areas that have been effectively captured based on the capture requirements and image registration error; removing the effectively captured areas from the original capture area to obtain uncaptured areas; and dividing the uncaptured areas according to the connectivity of the surface of the component to be inspected and counting the number of uncaptured areas. For surfaces with multiple shooting directions or independent key feature requirements, the corresponding shooting completion status is saved separately, and the fact that some surfaces have been effectively photographed does not replace all necessary shooting requirements.
[0009] In a preferred embodiment, determining the range of position change of the drainage line includes: converting the image position of the drainage line to the spatial coordinate system of the component to be inspected based on the acquisition time of the continuously captured images, the camera attitude and the position of the UAV; determining the position change rate and position change acceleration based on the continuous spatial position; and determining the spatial position determination error based on the camera calibration error, the UAV positioning error, the camera attitude error and the image feature matching error. The error is determined based on the space occupied by the drainage line during continuous shooting, the space that may be traversed between adjacent shooting moments, the shape and spatial position of the drainage line itself, and the range of positional variation of the drainage line is determined by combining the verified upper bound of motion. The uncertainty of the predicted position is determined based on the displacement that may be caused by the spatial position determination error and velocity estimation error within the prediction time, as well as the additional displacement corresponding to the verified upper limit of acceleration. The upper limit of the allowable predicted position uncertainty is determined based on the required clearance for the shooting line of sight, camera imaging error, and spatial measurement accuracy. The prediction time corresponding to when the predicted position uncertainty reaches the upper limit of the allowable predicted position uncertainty is taken as the effective prediction time. The applicable time for the position change range of the drainage line is determined based on the expected arrival time of the subsequent shooting task points. If a reliable upper bound on motion cannot be obtained, shorten the applicable time or reacquire continuous images with effective spatial constraints before the execution of subsequent shooting tasks.
[0010] In a preferred embodiment, obtaining the corrected shooting position includes: determining the estimated arrival time of the subsequent shooting task point based on the current actual position, the subsequent inspection route, the allowed flight speed and the shooting dwell time; and determining whether the observation path is blocked by the guide line based on the position change range corresponding to the estimated arrival time within the applicable time range of the guide line's position change. If the applicable time expires, either reacquire continuously captured images or retain the state that needs to be re-verified. The affected shooting task points are determined based on the minimum spatial distance between the observation path and the guide line and the imaging clearance. Available shooting positions are searched within the space that meets the shooting requirements and flight safety requirements. The distance is adjusted according to the position to determine the corrected shooting position, and other available shooting positions with different coverage relationships or subsequent connection distances are retained. After forming the revised subsequent inspection route according to the original visit order, the estimated arrival time is recalculated. When the recalculated estimated arrival time makes the range of positional changes corresponding to the original corrected shooting position no longer applicable or the observation path is obstructed again, the corrected shooting position and estimated arrival time are re-determined. If a consistent result cannot be reached, reacquire continuously captured images, shorten the planning time range, or retain the state that needs to be re-verified.
[0011] In a preferred embodiment, generating the first route and the second route includes: when there is only one unshot area and the corrected shooting position corresponding to the current shooting task point can completely shoot the unshot area, the current corrected shooting position is connected to the corrected subsequent inspection route to form the first route. Select available shooting locations from subsequent shooting task points and candidate shooting task points that have not yet been executed, which can fully capture the unshot areas, rearrange the access order of the unfinished tasks, and form a second route; When the corrected subsequent shooting task point can simultaneously complete the original shooting requirements and the reshooting of the unshot areas, it is visited only once and the shooting completion relationship is recorded separately.
[0012] In a preferred embodiment, generating the first route and the second route further includes: when there are multiple unshot areas, or when there is only one unshot area but a single shooting position cannot capture it completely, determining the unshot area and the corresponding surface portion that can be effectively captured by each available shooting position, and determining a combination of shooting positions that meets the coverage requirements of the unshot area based on the coverage union of each available shooting position. The corrected shooting position corresponding to the current shooting task point is combined with the available shooting position that can fill the remaining unshot area to form the first route; the available shooting positions that can jointly cover the unshot area are selected from the subsequent shooting task points and the candidate shooting task points that have not yet been executed, and the access order of the unfinished tasks is rearranged to form the second route. Retain available shooting locations with different coverage relationships, subsequent connection distances, or estimated arrival times for route comparison.
[0013] In a preferred embodiment, determining the updated inspection route includes: merging the areas that can be effectively photographed by each shooting position in the first route and the second route respectively, and checking the coverage requirements of the unphotographed areas, the preset number of visits, the original shooting requirements, flight safety requirements, remaining battery power and return-to-home constraints. Based on the same current actual location, remaining tasks and destination, the flight distances of the first route, the second route and the revised subsequent inspection route are accumulated respectively, and the new flight distance is calculated. When the revised subsequent inspection route cannot complete all the necessary shooting tasks, the original tasks that cannot be completed should be supplemented to form a common executable benchmark. When a common executable baseline cannot be established, the complete flight distances of executable candidate routes are compared under the same remaining mission requirements and endpoints, and the shorter distance of an unexecutable route is not used as the selection criterion. The estimated arrival time is recalculated based on the visit order of the first and second routes, and the applicable time and observation path of the location variation range of the diversion line are re-examined. When route rearrangement makes the corrected shooting position no longer meet the requirements, the corrected shooting position, coverage and route distance are re-determined until the corrected shooting position, expected arrival time and coverage relationship all meet the requirements, or the process of acquiring continuous shooting images and re-verifying is initiated. Among the candidate routes that meet all the necessary conditions, the route with the shorter new flight distance is selected as the updated inspection route.
[0014] In a preferred embodiment, updating the location change range of the unshot area and the drainage line includes: determining the area that has been effectively shot based on the supplementary images actually obtained after executing the updated inspection route, removing the area that has been effectively shot from the original shooting area, re-dividing the remaining unshot area and counting the number, retaining the completed shooting tasks, and not directly taking the expected coverage as the actual shooting completion result. The range of positional changes of the drainage line is re-determined based on continuously captured images. The spatial difference between the latest range of positional changes and the previous range of positional changes is compared. The change judgment threshold is determined based on spatial measurement error and image registration error. When the spatial difference exceeds the change judgment threshold, the shooting task points that have not yet been executed are re-judged using the latest position change range; If the change judgment threshold is not exceeded, the current route is maintained and sampling continues; for position changes that exceed the applicable time range, they are not used for deterministic occlusion judgment.
[0015] In a preferred embodiment, repeated position adjustment and route determination include: using the current actual position of the UAV as the starting point of the subsequent route, regenerating the first route and the second route based on the updated uncaptured area and the coverage relationship of each available capture location, recalculating the additional flight distance using the latest corrected subsequent inspection route as a common benchmark, and the route segments that have already been actually flown are no longer included in the current remaining route. When the range of positional variation of the drainage line is no longer applicable to the estimated arrival time of the subsequent shooting task points, reacquire continuous shooting images and determine the corrected shooting position; If an image that meets the shooting requirements still cannot be obtained after multiple position corrections, the system determines whether to continue shooting based on the preset allowed reshoot time, number of reshoots, and remaining battery power. When there is no route that meets the shooting requirements and flight safety requirements, the remaining battery power is insufficient to support reshooting and safe return, or continuous reshooting has not produced a new effective shooting area and the allowed reshooting time has been reached, or there are no other available shooting locations, stop the current reshooting attempt, retain the incomplete state of the unshot area and perform safety procedures. When all un-photographed areas are eliminated and each set of tasks meets the preset access quantity and original shooting requirements, the inspection task of the corresponding component to be inspected is determined to be completed.
[0016] The technical effects and advantages of the UAV inspection task planning method for power lines of this invention are as follows: Uncaptured areas are identified by actual captured images, and the range of positional changes of the diversion line is determined by combining continuous captured images. Obstruction judgment and position correction are performed on the observation paths of current and subsequent capture points, reducing subsequent repeated obstruction caused by only correcting the current capture point. Based on the corrected capture position, a first route for current offset re-capture and a second route for re-capture using subsequent capture points are generated. Under the condition of meeting the coverage requirements of uncaptured areas and the requirements of remaining capture tasks, the increased flight distance is compared using the corrected subsequent inspection routes as a common benchmark, making the flight costs of different re-capture schemes comparable and helping to reduce unnecessary detours and repeated flights. After updating the inspection route, uncaptured areas and corrected capture positions are re-determined based on actual re-captured images and changes in the position of the diversion line. Iterative planning is performed on unfinished tasks to reduce the risk of missed captures caused by dynamic obstruction, improving the reliability of power line inspection tasks and the adaptability of route planning. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of a method for planning unmanned aerial vehicle (UAV) inspection tasks for power lines according to the present invention.
[0018] Figure 2 This is a schematic diagram illustrating the range of positional variation of the drainage line and the correction of the shooting task point position in this invention.
[0019] Figure 3 This is a schematic diagram illustrating the impact of the number of uncaptured areas on subsequent processing. Detailed Implementation
[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0021] Example Please see Figures 1-3 As shown, this invention discloses a method for planning unmanned aerial vehicle (UAV) inspection tasks for power lines, including the following steps: Step 1: Select shooting task points that meet the preset number of visits, determine the visiting order based on the flight distance, and generate the initial inspection route; detect whether there are unshot areas on the parts to be inspected based on the images captured by the current shooting task points, and determine the range of position changes of the diversion line if there are unshot areas. Step 2: Based on the range of changes in the location of the un-photographed area and the diversion line, determine whether the observation path of the current and subsequent photography task points is blocked by the diversion line. Adjust the position of the affected photography task points to obtain the corrected photography position, and form the corrected subsequent inspection route according to the original visit order. Step 3: Based on the corrected shooting position, generate a first route for the subsequent tasks after the current shooting task point is offset and reshot, and a second route for supplementing the unshot areas and rearranging the unfinished tasks using the corrected subsequent shooting task points; compare the coverage of the unshot areas by the two routes and the additional flight distance relative to the corrected subsequent inspection route to determine the updated inspection route. Step 4: Update the inspection route, update the location changes of un-photographed areas and diversion lines, and repeat the location adjustment and route determination if un-photographed areas still exist.
[0022] In step one, select shooting task points that meet the preset number of visits, determine the visiting order based on the flight distance, and generate an initial inspection route; detect whether there are un-shot areas on the components to be inspected based on the images captured by the current shooting task points, and determine the range of position changes of the drainage line if un-shot areas exist. Specific details include: Candidate shooting task points are spatial locations that pre-meet the shooting distance, shooting direction, camera field of view and image resolution requirements of the parts to be inspected. Shooting task points include the spatial position of the UAV and the corresponding camera attitude. Multiple candidate shooting task points corresponding to the same component to be inspected constitute a set task. The preset number of visits is the number of shooting task points that the set task needs to visit during normal inspection. The preset number of visits is determined by the requirements of the corresponding inspection task and is not replaced by the number of unshot areas obtained by subsequent detection. Flight distance is the length of the flight path between two shooting mission points, provided that the flight safety requirements for the UAV are met for towers, power lines and other obstacles.
[0023] When generating the initial inspection route, the spatial coordinates of each candidate shooting task point, the corresponding component to be inspected, the set task to which it belongs, and the preset number of visits are obtained. From each set task, the shooting task point selection result that meets the preset number of visits is formed. Then, the visit order is determined according to the flight distance between the shooting task points, and the corresponding route is checked to see if it meets the drone's endurance constraints. The shooting task point selection result and visit order that meet the preset number of visits, flight safety requirements, and drone endurance constraints are determined as the initial inspection route.
[0024] For example, a genetic algorithm can be used to encode the set of tasks, shooting task points and access order, and candidate results can be generated through selection, crossover and mutation. Candidate results that do not meet the preset access quantity or drone endurance constraints can be corrected or eliminated. Alternatively, a local exchange method can be used to exchange two shooting task points in the access order, or a continuous segment of the access order can be reversed to redetermine the flight distance and retain the results that meet the constraints and have a smaller flight distance. It should be noted that the basic path optimization of the initial inspection route can adopt existing path optimization methods, and this part is used to form the basic route for subsequent dynamic occlusion processing.
[0025] The initial inspection route also saves the original shooting area and shooting requirements corresponding to each shooting task point; The original shooting area is the surface range of the component to be inspected that the current shooting task point needs to effectively acquire according to the inspection task requirements. It can be determined by the surface area in the 3D model of the component to be inspected, or by the standard image that has been registered to the component to be inspected. For components to be inspected that have a 3D model, the original shooting area can be saved according to the surface triangular mesh or the pre-marked component feature area; for components to be inspected that do not have a complete 3D model, the shooting area correspondence can be established based on the calibrated standard image and the feature points of the component to be inspected, so that images obtained from different shooting task points can be converted to the reference position of the same component to be inspected for comparison.
[0026] For example, for the component to be inspected that needs to check the connecting bolts, the bolt head, nut edge and connecting plate contact area can be respectively mapped to the original shooting area; for the component to be inspected that needs to check surface cracks or corrosion, the corresponding surface can be divided into a surface grid with spatial coordinates and area, and the shooting distance, shooting angle, image resolution and clarity requirements corresponding to each surface grid can be saved. Therefore, the original shooting area not only represents the outline of the component to be inspected that needs to appear in the shooting image, but also includes the specific surface content and key features required to complete the actual inspection task.
[0027] For each shooting task point, save the corresponding original shooting area, shooting requirements, areas that have been effectively shot, and shooting completion status; When the actual captured image meets the corresponding shooting requirements, update the area that has been effectively captured and the shooting completion status; when the shooting task point has been visited but there are still uncaptured areas, retain the uncaptured areas and the corresponding incomplete status, and use the uncaptured areas as input for generating subsequent reshoot routes. For the same component to be inspected, multiple shooting directions or multiple independent key features need to be checked. The completion status of each shooting direction and independent key feature should be recorded separately. The completion of one shooting direction or one key feature should not replace other necessary shooting requirements.
[0028] It should be noted that when the original shooting task points are replaced by corrected shooting locations or candidate shooting task points that have not yet been executed, it is still necessary to check whether the replaced set of tasks meets the preset number of visits and the original shooting requirements. For inspection tasks that allow additional photo shoots, the additional photo shoots and the original photo shoot requirements should be recorded separately. The increase in the number of photo shoots should not be used to arbitrarily offset the necessary access of other collection tasks.
[0029] The drone executes shooting task points according to the initial inspection route. When it arrives at the current shooting task point or passes through a position that meets the shooting requirements, it acquires shooting images according to the corresponding shooting direction, and records the image acquisition time, drone position, camera attitude and corresponding parts to be inspected. For tasks requiring stable shooting, the aircraft can hover briefly at the shooting location or reduce its flight speed; for tasks that can meet the shooting requirements during flight, image acquisition can be completed when passing the shooting task point.
[0030] When detecting whether there is an uncaptured area for the component to be inspected based on the captured images of the current shooting task point, the image area of the component to be inspected and the drainage line in the captured image is first identified. Then, based on the camera calibration parameters, the drone position, the camera attitude and the spatial position of the component to be inspected, the captured image is registered to the original shooting area. When the camera pose and the spatial position of the component to be inspected are known, the image pixels corresponding to the surface position of the component to be inspected can be determined according to the camera projection relationship; when a standard image is used, the image registration relationship can be determined according to the feature points of the component to be inspected in the standard image and the actual captured image.
[0031] After image registration is completed, it is determined one by one whether the surface of the component to be inspected in the original shooting area meets the effective shooting requirements; Effective shooting requirements include that the target surface is within the camera's field of view, the corresponding observation path is not obstructed by the drainage line, the image clarity meets the inspection requirements, and the necessary key features can be effectively acquired. For the surface of the component to be inspected that requires multiple shooting directions or multiple independent inspection requirements, the corresponding shooting completion status is saved separately, so that it is not considered that all shooting requirements have been completed based on the shooting image obtained from one shooting direction.
[0032] The image quality threshold is determined based on specific shooting requirements and camera calibration results. Regarding image resolution requirements, the minimum target feature size to be identified and the minimum number of recognizable pixels required for that target feature are determined based on the inspection task. Then, the object pixel size is determined based on the camera focal length, pixel size, and actual shooting distance. When the ratio between the minimum target feature size and the object pixel size is not less than the minimum number of recognizable pixels required, the corresponding surface is considered to meet the image resolution requirements; otherwise, the corresponding surface is not identified as an area that has been effectively captured.
[0033] For sharpness requirements, image edge gradient, Laplacian operator response, or other evaluation methods that can characterize the sharpness of the captured image can be used; First, obtain a calibration image that has been confirmed to meet the corresponding shooting requirements. Under the same camera, similar shooting distance, and the same image processing conditions, determine the sharpness reference range, and determine the sharpness judgment threshold based on the lower limit of the sharpness reference range. For different shooting distances or different lighting conditions, the corresponding sharpness reference range can be determined separately. When the sharpness evaluation value of the actual captured image is lower than the corresponding sharpness judgment threshold, the corresponding surface is not identified as an area that has been effectively captured.
[0034] Image registration error is also included in the determination of valid shooting; The image registration error can be determined based on the feature points of the component to be inspected or the calibration target with known spatial location, and the upper bound of the error at the corresponding shooting distance can be obtained; When the image registration error exceeds the allowable range that can reliably distinguish key features of the target, the image registration result is not used to directly confirm that the key area has been completed. Instead, the captured image is re-acquired or other reliable features are used for re-registration. If essential key features are obscured by the drainage line, even if the overall visible area of the component to be inspected is relatively high, the obscured essential key features will not be considered as having been effectively photographed.
[0035] Remove the areas that have already been effectively photographed from the original shooting area to obtain the unphotographed areas; For the implementation method that uses three-dimensional surface meshes to represent the original shooting area, the surface meshes that meet the effective shooting requirements are marked one by one, and the remaining surface meshes that do not meet the effective shooting requirements are determined as unshot areas; for the implementation method that uses standard images to represent the original shooting area, after registering the actual shooting image with the standard image, the image parts that do not meet the effective shooting requirements are mapped to the surface of the component to be inspected. For valid images of the same surface obtained from different shooting points, only one corresponding completion status is saved, and the image pixel position is not recalculated.
[0036] The un-photographed areas are divided into un-photographed areas based on their connectivity and the number of un-photographed areas is counted. For the implementation using surface meshes, adjacent meshes sharing the same surface boundary are identified as adjacent regions. A connectivity search is performed on the un-photographed surface meshes, and un-photographed surface meshes that are continuously connected through adjacency are classified into the same un-photographed region. For minor breaks caused by model discretization or image registration errors, the allowable connection distance can be determined based on the surface grid size and image registration error; for regions with independent key feature inspection requirements, merging should not cross independent component regions that have been confirmed to be validly captured; This yields the number of unphotographed areas and the specific surface extent of each unphotographed area.
[0037] When there are uncaptured areas, further continuous image captures are obtained to determine the range of positional changes of the drainage line; The continuous image capture includes at least multiple frames of images that can identify the relative position of the drainage line and the component to be inspected, and the image acquisition time, camera attitude and drone position corresponding to each captured image are recorded at the same time. By using the fixed features of the components or towers to be inspected, the continuously captured images are converted to the same spatial coordinate system, and then the spatial position of the diversion line at different shooting times is determined. The spatial coordinate system can be a local three-dimensional coordinate system with the fixed position of the tower as the origin, and all coordinate directions can use the same length unit.
[0038] The camera calibration parameters include focal length, principal point, camera attitude, and the positional relationship of the camera relative to the UAV positioning point. For the use of binocular cameras, the spatial position can be determined based on the corresponding positions of the same drainage line feature in the images of the two cameras and the known baseline between the cameras; for multi-view images obtained from different UAV positions, the corresponding observation rays can be transformed to the same spatial coordinate system, and the spatial position of the drainage line can be determined based on the spatial intersection relationship of the different observation rays. Connect the spatial points of the drainage line obtained at the same shooting time along the length of the drainage line to form the center line of the drainage line at the corresponding shooting time, and determine the actual space occupied by the drainage line at the corresponding shooting time by combining the shape of the drainage line itself.
[0039] It should be noted that, in cases where only monocular images are captured and effective depth constraints are lacking, the movement of pixel positions in a two-dimensional image is not directly considered as precise three-dimensional displacement. When the spatial location of the diversion line can be determined by combining the known fixed endpoints of the diversion line, the three-dimensional model of the line, or other reliable spatial constraints, the corresponding spatial constraints are used to solve for the location; when reliable spatial constraints are lacking, the uncertainty range of the spatial location is retained, and images with effective depth information or sufficient viewing angle difference are reacquired before subsequent shooting tasks are executed.
[0040] The spatial location determination error is determined based on the camera calibration error, UAV positioning error, camera attitude error, and image feature matching error. For binocular measurement, the spatial position can be repeatedly measured using a calibration target at a known distance to obtain the upper bound of the error at different measurement distances; for multi-view measurement, the error range can be determined based on the observation ray intersection residual and camera attitude error. When the correlation between different errors cannot be reliably determined, the sum of the upper bounds of each error can be used as a conservative spatial location determination error. When the spatial position determination error exceeds the spatial judgment accuracy allowed by the shooting requirements, the unobstructed position is not directly determined using the spatial position measurement results. Instead, the captured image is re-acquired, the viewing angle difference is increased, or the uncertainty range of the spatial position is expanded.
[0041] Based on the centerline of the drainage line at continuous shooting times, determine the corresponding centerline sampling positions along the length of the drainage line, calculate the spatial displacement and time interval of the corresponding centerline sampling positions at adjacent shooting times, and obtain the position change velocity; obtain the position change acceleration based on the difference between adjacent position change velocities and the corresponding time interval. For different centerline sampling positions, the corresponding position change results are saved separately, so that the sway of different parts of the drainage line participates in the determination of the subsequent position change range, instead of using the position change of one end of the drainage line to represent the entire drainage line.
[0042] It should be noted that the position change velocity and position change acceleration obtained from a finite number of historical samples are measurement estimates of the current motion state and are not directly used as the absolute upper bound of future motion. For situations requiring conservative space judgment, the upper limit of motion is determined based on the allowable physical swing range of the diversion line, line structure constraints, and the verified historical motion range under the same or similar operating conditions. Specifically, the upper bound of the position change velocity and the upper bound of the position change acceleration of each centerline sampling position can be determined separately, and the upper bound of the motion corresponding to different parts can be involved in determining the position change range of the drainage line. When a reliable upper bound for motion cannot be obtained, the applicable time for the positional variation range of the drainage line should be shortened, or continuous images with effective spatial constraints should be reacquired before the corresponding subsequent shooting task point is executed, and the limited historical sampling results should not be unconditionally extended to all subsequent inspection time.
[0043] The error is determined based on the space occupied by the drainage line during continuous shooting, the space that may be traversed between adjacent shooting moments, the shape and spatial position of the drainage line itself, and the range of positional variation of the drainage line is determined by combining the verified upper bound of motion. Specifically, a spatial range is formed around the center line of the drainage line at each shooting moment, including the shape of the drainage line itself and the error in determining its spatial position, and then expanded according to possible positional changes between adjacent shooting moments; For cases where a reliable upper bound on the velocity of position change has been obtained, the maximum possible displacement during the unsampled period is determined based on the time interval between adjacent shots and the upper bound on the velocity of position change. For cases where a reliable upper bound on the acceleration of position change is also available, the possible increase in displacement during the unsampled period is further determined based on the corresponding motion relationship. The expanded spatial range is then merged to form the position change range of the drainage line that includes the shape of the drainage line itself, the spatial position determination error, and the uncertainty of the unsampled motion.
[0044] For situations where the positional variation range of the drainage line needs to be used for subsequent shooting task points, the possible positional range of the subsequent shooting task points is calculated based on the acquisition time of the continuously captured images, the latest spatial position, and the corresponding upper limit of motion. The spatial position determination error and the shape of the drainage line itself are also included in the positional variation range of the corresponding drainage line.
[0045] Furthermore, based on the displacement that may be caused by the spatial location determination error and velocity estimation error within the prediction time, as well as the additional displacement that may be caused by the verified upper bound of the position change acceleration within the prediction time, the prediction position uncertainty is determined. Among them, the displacement that may be caused by the velocity estimation error is determined based on the velocity estimation error and the prediction time; the additional displacement that may be caused by the upper limit of the position change acceleration is determined based on the upper limit of the position change acceleration and the prediction time. The above uncertainties are combined to obtain the predicted location uncertainty for the corresponding prediction time.
[0046] The upper limit of the allowable uncertainty in the predicted position is determined based on the clearance required for the shooting line of sight, camera imaging error, and spatial measurement accuracy; Specifically, based on the shooting requirements of the component to be inspected, the effective imaging range that needs to be retained around the key features is determined, the corresponding image edge blank is converted into spatial distance, and the imaging net margin is determined in combination with the image registration error. Then, based on the imaging net margin and the spatial measurement accuracy that can be obtained, the upper limit of the allowable predicted position uncertainty that can be used for reliable occlusion judgment is determined. The upper limit of the allowable uncertainty in the predicted location is used to limit the reliability of the predicted location range of the diversion line, and is determined separately from the safe flight distance between the drone and the power line.
[0047] As the prediction time increases, the uncertainty of the predicted location gradually increases; When the uncertainty of the predicted position reaches the upper limit of the allowable uncertainty of the predicted position, the corresponding prediction time is taken as the effective prediction duration, and the applicable time of the position change range of the drainage line is determined according to the acquisition time of the latest continuously captured image and the effective prediction duration. For subsequent shooting locations whose expected arrival time is within the applicable time, the occlusion judgment is made using the position change range of the guide line corresponding to the expected arrival time; for subsequent shooting locations whose expected arrival time exceeds the applicable time, the position change range is not used to make a deterministic occlusion judgment, but instead, continuous shooting images are reacquired, the planning time range is shortened, or the state that needs to be re-verified is retained.
[0048] For example, the sampling interval for continuous image capture is 0.1 seconds, the upper bound of spatial position determination error is 0.02 meters, the upper bound of velocity estimation error is 0.1 meters per second, the upper bound of verified position change acceleration is 0.2 meters per square second, and the upper limit of allowable prediction position uncertainty is 0.12 meters. When the prediction time is 0.6 seconds, the spatial position determination error is 0.02 meters, the displacement caused by the velocity estimation error is 0.06 meters, and the additional displacement caused by the position change acceleration is 0.036 meters, with a total prediction position uncertainty of 0.116 meters, which does not exceed 0.12 meters. When the prediction time is 0.7 seconds, the corresponding total prediction position uncertainty is 0.139 meters, which exceeds 0.12 meters. Therefore, the corresponding effective prediction time can be controlled at approximately 0.6 seconds. The above values are used to illustrate the process of determining the effective prediction duration and do not constitute fixed parameters in actual lines.
[0049] To illustrate the subsequent processing, a local three-dimensional coordinate system can be established; Assume that the centers of the three independent shooting areas A, B and C of the component to be inspected are located at (0, -1, 0), (0, 0, 0) and (0, 1, 0) respectively, the current shooting task point 2 is located at (10, 0, 0), and the position of the drainage line changes near the plane with the horizontal coordinate 5. After spatial registration, the longitudinal position variation range of the centerline of the drainage line in the corresponding cross-section is -0.15 meters to 0.50 meters. After considering the shape of the drainage line itself, the spatial position determination error, the uncertainty of unsampled motion, and the imaging clearance, the longitudinal conservative coverage range of the corresponding position variation range is -0.25 meters to 0.60 meters.
[0050] The observation paths from the current shooting point 2 to the shooting areas A, B and C pass through -0.5 meters, 0 meters and 0.5 meters respectively on the plane with a horizontal coordinate of 5. Therefore, the observation path of the shooting area A is outside the position change range of the drainage line, while the observation paths of the shooting areas B and C are inside the position change range of the drainage line. When the actual captured image has met all the necessary shooting requirements of the area to be shot, while the key features of the areas to be shot, B and C, have not been fully obtained, the area to be shot, A, is recorded as the area that has been effectively captured, and the areas to be shot, B and C, are identified as two areas that have not been captured. For a certain area to be photographed, the actual judgment is still based on all the necessary surface grids and shooting requirements, not just the center coordinates mentioned above to determine whether the entire area is complete.
[0051] Step one ultimately obtains the initial inspection route, the area that has been effectively photographed corresponding to the current shooting task point, the area that has not been photographed, the number of areas that have not been photographed, the range of changes in the position of the diversion line and the corresponding applicable time, and saves the current actual position of the drone and the completion status of each collection task, and then passes the above results to step two.
[0052] In step two, based on the range of changes in the un-photographed areas and the location of the diversion line, it is determined whether the observation path of the current and subsequent photography task points is obstructed by the diversion line. The positions of the affected photography task points are adjusted to obtain the corrected photography positions, and the corrected subsequent inspection route is formed according to the original visit order. Specific details include: The current incomplete shooting task points, the subsequent shooting task points that have not yet been executed in the initial inspection route, and the candidate shooting task points that have not yet been executed are processed separately to obtain the corrected shooting positions that can be used for step three and the areas that can be effectively shot at each corrected shooting position. Among them, the current shooting task point is the shooting task point that the drone has reached and generated an area that has not been shot, the subsequent shooting task point is the shooting task point in the initial inspection route that has not yet been executed, and the candidate shooting task point that has not yet been executed is a candidate location that can be used to supplement shooting or replace subsequent tasks, but has not yet been marked as completed.
[0053] The current shooting task point is adjusted to supplement the unshooted area obtained in step one. Subsequent shooting task point adjustments are made to meet the original shooting area and shooting requirements. The areas that have been confirmed as valid for shooting in Step 1 will remain and will not be converted back to unshot areas due to changes in the location of the shooting task points.
[0054] The observation path is determined based on the location of the shooting task point, the camera posture, and the original shooting area of the corresponding component to be inspected; The observation path is the limited spatial path from the camera position to the corresponding position on the surface of the component to be inspected when the camera observes the surface of the component to be inspected. For a predetermined shooting area with a certain area, a corresponding observation path is formed based on all necessary surface grids, and each necessary observation path is used as an object for occlusion judgment. For the current shooting location, the main focus is on checking the observation path leading to the un-shooted area; for subsequent shooting locations, the focus is on checking the observation path leading to the corresponding originally designated shooting area.
[0055] The estimated arrival time of each subsequent shooting task point is determined based on the drone's current actual location, subsequent inspection route, permitted flight speed, and shooting dwell time. Specifically, the length of the flight path from the current actual location to the corresponding subsequent shooting task point is accumulated, the estimated flight time is determined according to the allowed flight speed of each path segment, and the necessary turning time and shooting dwell time at the previous shooting task point are added. When different flight path segments have different permissible flight speeds, the estimated flight time for each path segment is determined separately and then accumulated.
[0056] The estimated arrival time is used to determine the range of positional changes of the diversion line when the subsequent shooting tasks are actually carried out. For shooting task points whose expected arrival time is within the applicable time determined in step one, the path occlusion judgment is made by observing the range of position changes of the guide line corresponding to the expected arrival time; for shooting task points whose expected arrival time exceeds the applicable time, continuously captured images are re-acquired or the state that needs to be re-verified is retained, and a new occlusion judgment is completed before the corresponding shooting task point is executed.
[0057] When judging the spatial overlap between the observation path and the range of positional change of the drainage line, the camera position and the corresponding position of the surface to be photographed are connected to form a finite observation line segment, and then the minimum spatial distance between the observation line segment and the range of positional change of the drainage line is determined. For the range of positional changes represented by the centerline of the drainage line and the outer space, the minimum distance between the observation line segment and the centerline of the drainage line can be determined. Then, the shape of the drainage line itself, the spatial position determination error, the uncertainty of the predicted motion, and the imaging clearance are taken into account. When there is a clearance distance between the observation path and the drainage line that meets the imaging requirements, the corresponding observation path is determined to be unobstructed by the drainage line; when the clearance distance is less than or equal to the corresponding imaging clearance requirements, the corresponding observation path is determined to be affected by the obstruction of the drainage line.
[0058] The imaging clearance is determined based on the camera object-side pixel size, image registration error, and the edge margin required for key target features. Specifically, the effective imaging range that needs to be retained around the key features is determined according to the shooting requirements, the corresponding image edge white space is converted into spatial distance, and the imaging net clearance is determined in combination with the image registration error. Imaging clearance margin is used to determine whether the guide line may affect the effective imaging of the target area, and is determined separately from the safe flight distance of the UAV body relative to the line.
[0059] It should be noted that only when the positional change of the drainage line enters the corresponding observation path between the camera position and the component to be inspected will it cause occlusion of the corresponding surface to be photographed. If the drainage line is located behind the component to be inspected, or is only close to the component to be inspected in the two-dimensional image but does not enter the observation path in space, then it is not directly determined that the corresponding area to be photographed is blocked. For critical component areas that require complete imaging, check all necessary surface grids and observation paths of key features separately. Do not assume that the entire planned imaging area meets the requirements simply because some observation paths are not obstructed.
[0060] For the affected shooting locations, the allowable adjustment space is determined based on the spatial location of the component to be inspected, the allowable shooting distance, the camera field of view, the allowable shooting angle, and flight safety requirements. The allowable adjustment space is the area near the original shooting position that meets the corresponding shooting requirements, excluding spaces that do not meet flight safety requirements due to changes in the position of conductors, towers, guide lines, and other obstacles.
[0061] If adjusting the camera attitude alone can make the observation path avoid the range of changes in the position of the guide line and meet the shooting requirements, then retain the original spatial position of the drone and adjust the camera attitude; if there is still obstruction after adjusting the camera attitude, then search for a new drone shooting position within the allowable adjustment space. During the search, it is prohibited to conduct exploratory flights in the direction of the guide line that do not meet flight safety requirements to find and correct the shooting position.
[0062] In one possible implementation, the corrected shooting position is determined by combining finite candidate search with local refinement. First, based on the original shooting position, multiple candidate positions are generated within the allowable adjustment space, and the camera attitude that can meet the shooting requirements is determined for each candidate position. For each candidate position, the clearance distance between the corresponding observation path and the position change range of the guide line is determined, and the camera field of view, shooting distance, shooting angle, image resolution and flight safety requirements are checked. All candidate positions that meet the requirements are determined as usable shooting positions.
[0063] The spatial search interval can be determined based on the drone's position control accuracy, drone positioning error, and allowable change in shooting position. Candidate positions can be generated by first using a search interval that can cover the space that allows for adjustment. Then, the search interval can be narrowed for areas that are close to the occlusion boundary or have some usable positions. The corresponding observation path and shooting requirements can be re-evaluated until the search interval reaches the scale of position changes that the drone can stably distinguish and control, or further refinement no longer generates new usable shooting positions. The results obtained from a finite candidate search represent the available results among the searched candidate positions, and do not represent the discrete search results as the absolute optimal position in the entire continuous space.
[0064] In cases where there are multiple available shooting locations, the shooting location can be prioritized and corrected based on the flight distance between it and the original shooting location. At the same time, other available shooting locations with different coverage relationships or different subsequent connection distances are retained for the complete route comparison in step three. One should not delete another available shooting location that might result in a shorter overall distance for subsequent inspection routes simply because the local adjustment distance of a particular available shooting location is small.
[0065] For the current shooting location, record the unshot areas and specific surface parts that can be effectively shot from each available shooting position; for subsequent shooting locations, record the original shooting areas that can be completed from each available shooting position. When a modified shooting position can fully meet the original shooting requirements, it is used as a usable alternative position for the corresponding subsequent shooting task. When a single position can only complete part of the original shooting area, the part that can be completed is recorded, and other usable shooting positions or candidate shooting task points that have not yet been executed are searched for. Partial visibility is not mistakenly judged as the completion of the entire task.
[0066] For candidate shooting locations that have not yet been executed, the same spatial occlusion judgment and position adjustment method is used to determine the available shooting positions and areas that can be effectively shot for each candidate shooting location. When a candidate shooting location is blocked by a guide line, it is not directly and permanently eliminated. It will not be used as a current executable reshoot location only if there is no available shooting location that meets the shooting requirements and flight safety requirements within the allowable adjustment space, or if the corresponding location cannot be used to complete effective shooting within the current applicable time. The corresponding original task information will continue to be saved and can be re-evaluated after the guide line's position changes range is updated.
[0067] When forming the revised subsequent inspection route, the original order of visits to the points that have not yet been photographed in the initial inspection route is arranged. The photographing points that are affected by the obstruction of the diversion line are replaced with the corresponding corrected photographing positions, and the photographing points that are not affected remain in their original positions. If there are multiple available shooting locations for the same subsequent shooting task point, the corrected subsequent inspection route can be formed by first using the available shooting locations with smaller adjustment distances, and then saving the available shooting locations with other coverage relationships or subsequent connection distances for use in step three.
[0068] It should be noted that the revised subsequent inspection route does not change the already completed shooting tasks, nor does it automatically reduce the preset number of visits for the collection task. When a certain corrected shooting position can simultaneously complete the original subsequent shooting tasks and the reshooting of the currently unshot areas, it can be visited only once in the subsequent route, but the corresponding task completion relationship is recorded separately. When replacing the original subsequent shooting points with other candidate shooting points, the preset number of visits and the original shooting requirements of the replaced set of tasks are still checked.
[0069] Since changes in the location of subsequent shooting mission points will alter the flight distance and estimated arrival time of subsequent routes, the estimated arrival time of each subsequent shooting mission point is recalculated after the revised subsequent inspection route is formed. First, the estimated arrival time is calculated based on the current actual location and the shooting task points that have not yet been executed, according to the original visit order. Then, the shooting position is determined according to the range of position change of the diversion line corresponding to the estimated arrival time. Finally, a new and corrected subsequent inspection route is formed based on the corrected shooting position, and the estimated arrival time is recalculated.
[0070] After recalculating the estimated arrival time, check one by one whether the position change range of the diversion line corresponding to each subsequent shooting task point is still within the applicable time, and re-determine whether the observation path corresponding to the shooting position is corrected to avoid the position change range of the diversion line. When the new estimated arrival time makes the position change range corresponding to the original corrected shooting position no longer applicable, or when the position change range corresponding to the estimated arrival time re-enters the observation path, the corrected shooting position of the shooting task point is re-determined, and the estimated arrival time of the shooting task point and subsequent shooting task points is recalculated based on the re-determined corrected shooting position.
[0071] When the location of a subsequent shooting task point changes, the estimated arrival time of that shooting task point and subsequent shooting task points is recalculated. If the change in the estimated arrival time further renders the range of location changes corresponding to other subsequent shooting task points no longer applicable, the affected subsequent shooting task points are re-verified, instead of only checking the shooting task points whose locations have changed.
[0072] When the corrected shooting position, recalculated estimated arrival time, and corresponding observation path all meet the current applicable conditions, the corresponding corrected subsequent inspection route shall be retained. In cases where a consistent result cannot be formed within the current effective prediction time, continuously captured images should be acquired again, the current planning time range should be shortened, or the state that needs to be re-verified should be retained. Corrected shooting positions that have not yet met the time applicability conditions should not be directly used as determined executable positions.
[0073] For example, using the simulation conditions of step one, the current shooting task point 2 is located at (10, 0, 0), the shooting areas B and C are unshooting areas, and the position variation range of the drainage line is conservatively within the longitudinal coverage range of -0.25 meters to 0.60 meters in the corresponding cross-section. When the current shooting task point 2 is adjusted to (10, -2, 0), the observation paths to areas B and C to be shot can both avoid the range of position change and meet the requirements of camera field of view, shooting distance, shooting angle and flight safety. Then, this position is determined as a corrected shooting position of the current shooting task point 2.
[0074] Assuming that the subsequent shooting task point 3 is located at (10, 1, 0), the observation path of its corresponding original shooting area enters the position change range of the guide line. After allowing adjustment of the spatial search, (12, 2, 0) is determined to be a usable shooting position. Furthermore, the observation path from this position to the corresponding shooting area is all outside the position change range of the current guide line. When shooting requirements and flight safety requirements are met, (12, 2, 0) is determined as the corrected shooting position of the subsequent shooting task point 3. The other subsequent shooting location, which is not affected by the current diversion line, will remain in its original position; The spatial coordinates mentioned above are only used to illustrate the process of determining the shooting position and coverage relationship. In actual application, they should be determined according to the specific components to be inspected and the space of the line.
[0075] Step two ultimately obtains the corrected shooting position corresponding to the current shooting task point, the corrected shooting position corresponding to the subsequent shooting task point, the available shooting position corresponding to the candidate shooting task point that has not yet been executed, the area that each available shooting position can effectively shoot, and the corrected subsequent inspection route, and then passes the above results to step three.
[0076] In step three, based on the corrected shooting position, a first route is generated to continue the subsequent tasks after offsetting and reshooting the current shooting task point, and a second route is generated to supplement the unshot areas and rearrange the unfinished tasks using the corrected subsequent shooting task points. The coverage of the unshot areas by the two routes and the increased flight distance relative to the corrected subsequent inspection route are compared to determine the updated inspection route. The specific content includes: Based on the unshot areas determined in step one and the areas that can be effectively shot from each available shooting location determined in step two, establish the coverage relationship between the available shooting locations and the unshot areas. For the implementation using surface mesh representation, each uncaptured surface mesh is associated with each available capture position. When the corresponding available capture position meets the capture requirements and the corresponding observation path is not obstructed by the drainage line, the corresponding surface mesh is determined as the area that can be effectively captured at that available capture position. If the capture requirements are not met or the observation path is obstructed, the corresponding surface mesh is not counted as the effective coverage of that position.
[0077] For multiple surface grids that can be covered by an available shooting location, they are merged to obtain the effective coverage area of that available shooting location; when multiple available shooting locations cover the same surface grid, overlapping surface grids are only counted once when judging the route coverage result; For unshot areas with necessary key features or multiple shooting directions, in addition to determining the surface coverage relationship, save each key feature and whether the shooting direction meets the requirements. Only when all available shooting locations included in the route collectively meet all necessary surface grid, necessary key features, and shooting direction requirements is the corresponding route determined to meet the unshooting area coverage requirement. A higher surface coverage rate does not directly replace the necessary key feature requirements.
[0078] When there is only one uncaptured area and the corrected capture position corresponding to the current capture task point can completely capture the uncaptured area, the current corrected capture position is connected to the corrected subsequent inspection route to form a candidate result for the first route. At the same time, from the corrected subsequent capture task points and the candidate capture task points that have not yet been executed, the available capture position that can completely capture the uncaptured area is determined, and the corresponding available capture position and the unfinished task are used together to re-determine the access order to form a candidate result for the second route.
[0079] When there is only one uncaptured area, but no single available shooting position can completely capture that uncaptured area, shooting position combinations are formed based on the specific surface parts that can be effectively captured by different available shooting positions. If the corrected shooting position corresponding to the current shooting task point can effectively capture a portion of the area, the first route retains the corrected shooting position corresponding to the current shooting task point and selects other available shooting positions to supplement the remaining surface area; the second route selects available shooting positions from the corrected subsequent shooting task points and the candidate shooting task points that have not yet been executed, which can jointly complete the unshot area.
[0080] When there are multiple unshot areas, determine the unshot areas and specific surface parts that can be shot from each available shooting position. First, determine whether there is an available shooting position that can cover all unshot areas at the same time. If so, use the corresponding position as a candidate for single-point reshoot. If not, form different shooting position combinations based on the specific surface parts that each available shooting position can cover. For the first route, the corrected shooting position corresponding to the current shooting task point is retained in the shooting position combination, and an available shooting position that can supplement the remaining unshot areas is selected; for the second route, an available shooting position that can jointly cover all unshot areas is selected from the corrected subsequent shooting task points and the candidate shooting task points that have not yet been executed.
[0081] It should be noted that the number of unshot areas is used to determine the coverage relationship and is not directly used as the basis for judging the number of reshot locations; When an available shooting location can cover multiple unshootable areas, it will not be counted repeatedly as an available shooting location due to the large number of unshootable areas; when an unshootable area requires multiple available shooting locations to cover, the combination of multiple available shooting locations will not be prohibited simply because the number of unshootable areas is 1. When a corrected subsequent shooting point can simultaneously complete the original shooting requirements and reshoot areas that were not currently being shot, it is visited only once in the route, and the corresponding task completion relationship is recorded.
[0082] In one possible implementation, a set-coverage search can be used to form a combination of replacement shooting positions; All uncaptured surface grids are identified as the surface range that needs to be covered. The surface grids that can be covered by each available shooting location and the corresponding shooting requirements are read. For cases where the number of available shooting locations is limited, different combinations of shooting locations can be enumerated, and the surface grids that each combination can cover can be combined into a union. The combination that can cover all necessary surface grids and meet the requirements of key features and shooting direction can be retained.
[0083] When there are many available shooting locations, you can first select the available shooting location that can increase the coverage area of the currently unshot area, update the remaining unshot area, and then continue to select other available shooting locations until all necessary surface grids are covered or there are no more available shooting locations that can increase coverage. When using greedy search to form candidate combinations, the candidate selection order can be determined based on the newly added surface area of an available shooting location and the increased flight distance when it is added to the current route; when the newly added coverage areas are the same, the location with the smaller increase in flight distance is given priority; when the newly added coverage areas are different, the newly added coverage area corresponding to a unit of increased flight distance can be compared. This selection process is used to generate candidate combinations and does not directly replace subsequent comparisons of complete coverage and additional flight distances.
[0084] The combination obtained by greedy search is not necessarily the globally optimal combination among all available shooting positions. New candidate combinations can be formed by deleting redundant positions, replacing shooting positions, or adjusting the access order. If deleting a certain available shooting location can still cover all unshot areas and the remaining task requirements are not affected, the corresponding redundant location can be deleted; if replacing a shooting location with another available shooting location still satisfies the full coverage requirements and the remaining task requirements, and the additional flight distance is small, the replacement result should be retained. If the number of candidate locations can be fully enumerated, all combinations that meet the conditions can be enumerated; when using heuristic search, candidate routes that meet the conditions and have a smaller route distance are determined from the searched candidate results.
[0085] For available shooting locations with different coverage relationships, filtering can be performed without changing the necessary task constraints; The corresponding replacement position will be deleted only if a certain available shooting position can cover the entire necessary surface area of another available shooting position, and the flight distance between the available shooting position and the possible preceding and following positions is no greater than that of the other available shooting position under the same remaining mission conditions, and at the same time, the flight safety requirements and time applicability conditions are not reduced. If two locations have the same coverage but different subsequent connection distances or different estimated arrival times, they will be retained for comparison of subsequent complete routes.
[0086] When forming the first route, the current actual position of the drone is taken as the starting point, and the corrected shooting position corresponding to the current shooting task point is taken as the priority shooting position. After completing the unshot area that can be effectively shot, it is connected to the corrected subsequent inspection route. If there is still an unshot area, other available shooting positions that can fill the remaining unshot area are inserted into the subsequent inspection route, and the access order is determined according to the executable flight relationship.
[0087] When forming the second route, the current actual position of the drone is also taken as the starting point. Instead of taking additional shots near the current shooting task point, the available shooting positions that can fill the unshot areas from the corrected subsequent shooting task points and the candidate shooting task points that have not yet been executed are added to the locations to be visited, and the visiting order is determined together with the original subsequent unfinished tasks. When a corrected subsequent shooting point can simultaneously complete the corresponding original shooting requirements and the reshoot of the currently unshot area, the location is visited only once in the second route, and the corresponding completion relationship is recorded respectively.
[0088] The first route, the second route, and the revised subsequent inspection routes all use the same current actual location of the drone as the starting point, the same remaining task requirements, and the same endpoint. For cases where only a portion of the subsequent routes are rearranged, a common endpoint can be selected as a point where the first and second routes rejoin. For cases where all remaining inspection tasks are compared, the return point of this inspection task can be used as the common endpoint.
[0089] The common remaining task requirements include the original shooting requirements that have not yet been completed, the preset number of visits for each set of tasks, and all unshot areas that need to be supplemented. Different routes may use different modified shooting positions or alternative shooting points, and may have different visit sequences, but a route cannot gain an incomparable distance advantage by reducing necessary shooting tasks or changing the common endpoint.
[0090] When calculating the new flight distance, the length of the flyable path between adjacent shooting positions in the first route, the second route and the corrected subsequent inspection route are accumulated respectively. When inserting a reshoot location between two adjacent subsequent shooting task points, add the flight distance from the previous shooting location to the reshoot location and the flight distance from the reshoot location to the next shooting location, and then subtract the original flight distance between the two adjacent shooting locations to obtain the corresponding new flight distance. When inserting multiple reshoot locations or rearranging all remaining tasks, accumulate the length of the rearranged complete route and subtract the length of the corrected subsequent inspection route to obtain the corresponding new flight distance.
[0091] The flight distance between adjacent shooting positions can be determined based on existing three-dimensional safe path planning methods. When there are no obstacles and straight-line flight meets flight safety requirements, the straight-line distance between the two locations can be used; when there are poles, conductors, terrain, or other obstacles, the actual flyable path length that meets flight safety requirements should be used. When different flight directions have different flight restrictions or flight distances, the flight distances for the corresponding directions are saved separately, without assuming that the round-trip flight distances are the same.
[0092] The added flight distance can be a positive value, zero, or a negative value; When a certain reshoot location simultaneously replaces the original subsequent shooting task point and shortens the remaining route, the additional flight distance of the corresponding route relative to the corrected subsequent inspection route can be less than 0. The subsequent positional corrections caused by changes in the diversion line's position, which all candidate routes must undertake, are already included in the corrected subsequent inspection routes and will not be counted again in the additional flight distance of a particular reshoot scheme.
[0093] It should be noted that when using the same corrected subsequent inspection route as a comparison benchmark, the order of the newly added flight distances is consistent with the order of the complete flight distances of the corresponding candidate routes. This invention does not use distance subtraction itself as a necessary technical means to distinguish it from existing path optimization methods. Instead, it first performs a unified occlusion judgment and position correction on the current and subsequent shooting task points based on the position change range of the diversion line. After the route is rearranged, the expected arrival time, the position change range of the diversion line, and the observation path are re-examined so that the candidate routes participating in the route comparison have the same remaining task requirements and actual executable conditions.
[0094] If the revised subsequent inspection route cannot complete all the necessary shooting tasks on its own, the route will not be directly used as a complete and executable route for comparison. First, identify the subsequent shooting task points in the revised subsequent inspection route that cannot yet meet the original shooting requirements. Then, select available shooting locations from the candidate shooting task points that have not yet been executed to supplement the original shooting requirements. Check the preset number of visits, original shooting requirements, flight safety requirements, remaining battery power and return constraints of the supplemented group task to form a common executable benchmark that can complete all necessary tasks.
[0095] When a common executable baseline cannot be formed, the complete flight distances of each executable candidate route are compared under the same remaining mission requirements and the same endpoint conditions. The status of missions that cannot be completed is saved, and the shorter route that cannot complete the necessary shooting mission is not used as the selection criterion.
[0096] Before determining the updated inspection route, first check whether the first and second routes meet the requirements for coverage of unphotographed areas, the preset number of visits for the assembly task, and the original photography requirements. Then check the flight safety requirements, remaining battery power, and return constraints. For remaining power constraints, the estimated remaining route energy consumption can be determined based on the expected flight time, flight status, and verified energy consumption relationships of the candidate routes, and then judged in conjunction with the energy consumption required for a safe return and the necessary energy margin. A candidate route will only be retained as an executable candidate route if the sum of the estimated remaining route energy consumption and the energy consumption required for a safe return does not exceed the current available power of the drone.
[0097] The safe flight distance, remaining battery power, and permissible flight speed are determined based on the specific drone, line operation requirements, and flight environment, and do not adopt arbitrary general fixed values that are unrelated to specific equipment.
[0098] Since the first and second routes may change the access order of subsequent shooting task points, after the candidate routes are generated, the estimated arrival time of subsequent shooting task points is recalculated according to the respective access order of the first and second routes. Then, the applicable time and observation path of the corresponding drainage line position change range are rechecked according to the recalculated estimated arrival time.
[0099] If the route rearrangement causes the estimated arrival time of a certain shooting task point to exceed the applicable time of the original diversion line's position change range, then continuously shooting images will be acquired again or the corresponding shooting task point will be marked as needing re-verification. The original corrected shooting position that met the occlusion requirements will not be directly used to the new execution time. If the new estimated arrival time is still within the applicable time, but the positional change range of the drainage line corresponding to that time re-enters the observation path of the corrected shooting position, then the corrected shooting position of the shooting task point is re-determined.
[0100] After re-determining and correcting the shooting location, the coverage of the corresponding candidate routes, the complete flight distance, the newly added flight distance, and the estimated arrival time of subsequent shooting task points are recalculated. The coverage requirements of the unshooting areas, the preset number of visits, the original shooting requirements, the flight safety requirements, the remaining battery power, and the return-to-home constraints are checked again. If the corrected shooting location, estimated arrival time, and coverage relationship all meet the requirements, the corresponding candidate route is retained; if the requirements are still not met, the corrected shooting location of the affected shooting task point is determined again or the candidate route is eliminated.
[0101] The recalibrated route covers all subsequent shooting locations whose estimated arrival time is affected, not just newly added reshoot locations; If the corrected shooting position of a certain subsequent shooting task point changes and causes the subsequent visit time to continue to change, the estimated arrival time of the subsequent shooting task point is recalculated, and the applicable time and observation path of the corresponding diversion line position change range are re-determined. If an executable route cannot be formed within the current effective forecast time, shorten the current planning time range or reacquire continuous images before subsequent execution.
[0102] Among the candidate routes that meet all the necessary conditions, the route with the shorter new flight distance is selected as the updated inspection route; when both the first and second routes meet the conditions and the new flight distance is the same, further selection can be made based on the expected flight time, the adjustment distance of the current corrected shooting position, or other determined task constraints; when only one type of route meets all the conditions, the route that meets the conditions is selected; when neither type of route can produce an executable result, the unshot area is not marked as completed, and the process proceeds to step four, which involves reacquiring continuous shooting images or handling safety issues.
[0103] For example, following the simulation implementation of steps one and two, the current actual position is (10, 0, 0) corresponding to shooting task point 2, the corrected shooting position of subsequent shooting task point 3 is (12, 2, 0), and subsequent shooting task point 4 remains at (10, 6, 0). Assuming that the straight-line flight in the above space meets the flight safety requirements, the distance from the current actual position to the corrected shooting position of the subsequent shooting task point 3 is about 2.83 meters, the distance from the corrected shooting position to the subsequent shooting task point 4 is about 4.47 meters, and the cumulative length of the corrected subsequent inspection route is about 7.30 meters.
[0104] The first route moves from the current actual position to the corrected shooting position (10, -2, 0) corresponding to the current shooting task point, completes the supplementary shooting of the unshot area, and then proceeds to the corrected shooting position of subsequent shooting task point 3 and subsequent shooting task point 4 in sequence. The total length of the corresponding route is approximately 10.94 meters, and the additional flight distance compared to the corrected subsequent inspection route is approximately 3.64 meters. The second route does not perform offset reshoots near the current position, but directly reaches the corrected shooting position of the subsequent shooting task point 3. At this position, the original subsequent shooting task and the reshoot of the currently unshot area are completed at the same time, and then the route proceeds to the subsequent shooting task point 4. The length of the second route is approximately 7.30 meters, and the additional flight distance is 0. When both routes meet the conditions of complete filming, preset number of visits, flight safety, remaining battery power, and applicable time, the second route is determined as the update inspection route.
[0105] The coordinates, coverage relationships, and route distances mentioned above are all simulated data to illustrate the algorithm's processing. In actual applications, a flight path is established based on the specific tower, diversion line, component to be inspected, and flight safety space, and candidate routes are compared under the same shooting requirements, the same remaining tasks, and the same safety constraints.
[0106] Step 3 finally obtains the updated inspection route and saves the corresponding shooting location, access order, parts to be inspected, expected unshooting areas that can be covered, expected new flight distance and corresponding expected arrival time; for shooting task points that still need to be re-verified, the applicable time of the corresponding location change range and the status that needs to be re-verified are saved. Step four determines the task completion status based on the images obtained after the actual execution of the updated inspection route, and does not directly use the expected coverage result from step three as the actual shooting completion result.
[0107] In step four, the inspection route is updated, and the location changes of un-photographed areas and drainage lines are updated. If un-photographed areas still exist, the location adjustment and route determination are repeated. Specific details include: Perform supplementary shooting and subsequent inspection tasks according to the updated inspection route determined in step three. Update the areas that have been effectively shot based on the actual shooting results, and re-determine the shooting task points that have not yet been executed based on the latest changes in the location of the diversion line, so as to form the input required for the next location adjustment and route determination.
[0108] The drone sequentially executes the corresponding shooting positions according to the updated inspection route, and records the actual flight position, actual captured images, image acquisition time, camera attitude, and corresponding components to be inspected; For reshoot locations, images are acquired based on the shooting requirements of the corresponding unshot areas; for normal subsequent shooting locations, images are acquired based on the shooting requirements of the corresponding originally planned shooting areas. After the drone reaches the reshoot location, the guide line continues to change position, causing the corresponding inspection component to still not meet the shooting requirements. Only the part that has been effectively shot is recorded, and the remaining unfinished surface is retained as the unshot area.
[0109] When updating uncaptured areas based on actual recaptured images, the same image registration and valid capture judgment method as in step one is used to convert the areas that have been effectively captured in the recaptured images to the unified reference position of the parts to be inspected, and then merge them with the areas that have been effectively captured previously. The merged area that has already been effectively captured is removed from the original capture area to obtain the updated uncaptured area; In cases where the same surface of a component to be inspected is repeatedly photographed from different shooting points, only the corresponding valid shooting completion status is saved, and the corresponding surface area is not repeatedly accumulated.
[0110] It should be noted that the expected coverage determined in step three cannot replace the actual shooting results; If a certain available shooting location is expected to be able to completely capture the uncaptured area during the planning stage, but the actual captured image does not meet the shooting requirements due to new drainage lines blocking, blurry imaging, or changes in camera posture, the corresponding surface will still be retained as the uncaptured area. The shooting completion status is only updated when the actual captured images meet the corresponding shooting requirements; areas that have been confirmed as validly captured will not be automatically restored to an incomplete status due to subsequent replanning.
[0111] Based on the updated unshot areas, the independent unshot areas were redefined and the number of unshot areas was counted. For the implementation using surface mesh representation, the surface meshes that have been effectively captured are removed from the original shooting area, and the remaining surface meshes are subjected to connectivity search according to surface adjacency to obtain the updated uncaptured area; When the middle part of a previously unphotographed area has been completed, causing its remaining surface to be divided into multiple areas, the area is re-divided according to the actual surface connectivity. When all unphotographed areas have been completed, the number of unphotographed areas is reduced to zero. When only some unphotographed areas have been completed, only the remaining areas and their corresponding photography requirements are retained.
[0112] For example, if a component to be inspected originally has un-photographed areas A, B, and C, and the first re-photographing completes part of A and B, but C is still blocked by the drainage line, then the parts of A and B that have been effectively photographed are removed from the original photographing areas, and the remaining part of B and C are re-determined as the current un-photographed areas. Subsequent route planning will no longer require shooting the already completed A, but will instead determine the reshoot locations based on the remaining part of B and the coverage relationship between the available shooting locations corresponding to C; If the same candidate shooting point can simultaneously cover the remaining part of B and C, then it is used as a single-point supplementary shooting candidate; if it cannot cover them simultaneously, a combination of shooting positions that can jointly cover the remaining unshot area is formed.
[0113] At the same time, the range of positional changes of the drainage line is re-determined based on the images continuously acquired during the execution of the updated inspection route; Update the spatial position of the center line of the drainage line according to the spatial position determination method in step one, and determine the latest range of position change of the drainage line by combining the new continuous spatial position, spatial position determination error, possible movement between adjacent shooting times and the corresponding upper limit of movement.
[0114] Historical location changes that have exceeded the current applicable time will not be used for determining occlusion in subsequent shooting tasks; measurement results that are still within the current continuous motion process and within the applicable time can be retained and used to determine the latest location change range.
[0115] Compare the spatial differences between the latest range of positional change and the previous range of positional change; Spatial differences can be determined based on the spatial distance between the boundaries of the two positional change ranges, or based on the spatial range corresponding to the non-overlapping parts between the two positional change ranges, and the change judgment threshold can be determined based on spatial measurement error and image registration error.
[0116] In one possible implementation, multiple repeated measurements are performed when the drainage line is relatively stable or the shooting conditions are stable. The upper limit of normal measurement fluctuation is determined based on the spatial position differences obtained under the same or similar shooting distance, camera posture and image processing conditions. The upper limit of normal measurement fluctuation is used as the basis for determining the change judgment threshold. For different shooting distances or different camera postures, the upper limit of the corresponding normal measurement fluctuation can be determined, or the change judgment threshold can be adjusted according to the verified spatial position measurement error relationship, thereby reducing the impact of measurement noise on the judgment of position change.
[0117] When the spatial difference between the latest position change range and the previous position change range exceeds the change judgment threshold, it is determined that the position change range of the drainage line has changed effectively, and the position change range of the latest drainage line is used to re-judge the shooting task points that have not yet been executed; when the spatial difference does not exceed the change judgment threshold, the current route can be maintained and the shooting images can continue to be acquired, without repeatedly triggering shooting position adjustments and route rearrangement due to normal spatial measurement fluctuations.
[0118] The fact that the spatial difference does not exceed the change judgment threshold only indicates that the current measurement has not confirmed an effective change in the position change range of the drainage line, and does not mean that the applicable time of the original position change range of the drainage line can be extended; When the original drainage line's position change range has exceeded the applicable time, even if the spatial difference between the latest position change range and the previous position change range does not exceed the change judgment threshold, the position change range that has exceeded the applicable time will not be used to make a deterministic unobstructed judgment. Instead, continuously captured images will be acquired again or the state that needs to be re-verified will be retained.
[0119] After redetermining the range of changes in the location of the diversion line, the current actual location of the drone is used as the starting point of the subsequent route. The unexecuted shooting task points and unexecuted candidate shooting task points are re-acquired, and the corresponding observation path is re-determined according to step two to see if it is blocked by the diversion line. For subsequent shooting points whose latest position change range still applies to the estimated arrival time, the position change range of the guide line corresponding to the estimated arrival time is used for judgment; for shooting points whose estimated arrival time has exceeded the applicable time, the state that needs to be re-verified is retained, and continuous shooting images are re-acquired before the corresponding shooting point is executed; for shooting points that are affected again, the corrected shooting position is re-determined, and the original corrected shooting position that has become invalid is not continued to be used.
[0120] When there is only one unshot area after the update, follow the processing method for a single unshot area in step three, determine whether the corrected shooting position corresponding to the current shooting task point can completely shoot the corresponding unshot area, and compare it with the reshoot route formed by the corrected subsequent shooting task points or the candidate shooting task points that have not yet been executed. When a single available shooting location can capture the entire area, a single-point reshoot candidate is formed; when a single available shooting location can only cover part of the surface, a combination of shooting locations is formed based on the coverage relationship of the remaining surface, without forcing an unshot area to correspond to a single reshoot location.
[0121] When there are multiple unshot areas after the update, determine the unshot areas that can be covered by each available shooting position. First, determine whether there is an available shooting position that can cover all unshot areas at the same time. If not, form a combination of shooting positions based on the specific surface parts that can be covered by different available shooting positions, and determine whether the union of the areas covered by each shooting position meets the shooting requirements of all unshot areas. Areas that have already been completed will no longer be included in new shooting location combinations. The original combination of reshoot locations for all unshot areas can be regenerated based on the latest actual shooting results. Reshoot locations that are no longer needed will not be continued simply because the original route has been planned.
[0122] For example, following the simulation implementation of steps one to three, after the drone arrives at the corrected shooting position of the subsequent shooting task point 3 according to the second route, the actual shooting image completes the unshot area B, while the unshot area C still does not meet the shooting requirements due to the new position change of the diversion line. Therefore, B is removed from the unshot area, only C is retained, and the position change range of the diversion line is re-determined according to the latest continuous shooting images. Since the number of remaining unshot areas has decreased from 2 to 1, the reshoot route will be regenerated according to the handling method for a single unshot area, and the original reshoot requirements for both B and C will not be executed.
[0123] If the new continuous images show that the position of the drainage line has changed, causing the observation path from the original corrected shooting position to the unshot area C to be blocked again, then follow step two to search for a new corrected shooting position in the allowable adjustment space near the current actual position; at the same time, determine other available shooting positions that can meet the shooting requirements of the unshot area C from the candidate shooting task points that have not yet been executed, thereby re-forming the first route and the second route, and re-compare the corresponding coverage and the newly added flight distance according to step three.
[0124] When regenerating the first and second routes, the current actual position of the drone is used as the starting point of the new subsequent routes. The combination of supplementary shooting positions is re-determined based on the updated unshot areas and the coverage relationship of each available shooting position. The newly revised subsequent inspection route is used as a common benchmark to recalculate the additional flight distance. Route segments that have already been flown are no longer counted in the current remaining route, and completed shooting mission points are no longer considered shooting mission points that must be revisited.
[0125] For corrected shooting positions that have been redefined due to changes in the location of the latest diversion line, the estimated arrival time and flight distance of the corresponding subsequent shooting task points are recalculated. The original corrected shooting positions, the previous estimated arrival time, or the previously planned route distance that have become invalid are no longer used. When the new corrected shooting position changes the expected arrival time of subsequent shooting task points, continue to follow steps two and three to re-determine the applicable time and observation path for the corresponding range of position changes.
[0126] If a satisfactory image cannot be obtained after multiple position corrections, the decision to continue shooting is made based on the preset allowable reshoot time, number of reshoots, and remaining battery power. The allowed reshoot time is determined based on the inspection task plan, remaining task time, and drone endurance; the allowed number of reshoots is determined based on the allowed reshoot time and the shortest executable time required for one reshoot; the remaining power constraint is determined based on the expected energy consumption of the remaining route and the energy consumption required for a safe return.
[0127] When there is no feasible route that meets both shooting requirements and flight safety requirements, the system will not continue to generate a reshoot route that does not meet safety requirements. Instead, it will retain the incomplete state of the unshot area and implement safety measures. When the drone's remaining battery power is insufficient to support continued reshooting and a safe return, the current reshooting process will be stopped. If continuous reshoots fail to produce new areas that have already been effectively captured, and the allowed reshoot time has been reached or there are no other available shooting locations, the current reshoot attempt will also be stopped.
[0128] When the position of the guide line changes continuously, making it impossible for all currently available shooting positions to meet flight safety requirements, instead of completing the shooting by lowering safety requirements or forcibly approaching the guide line, the drone is kept in a safe state, returns to base, or the corresponding unfinished task is handed over to manual review.
[0129] When all un-photographed areas are eliminated, and the preset number of visits and original shooting requirements are met for each set of tasks, the inspection task for the corresponding component to be inspected is determined to be completed; if there are still unfinished tasks for other components to be inspected, the subsequent shooting task points will continue to be executed according to the currently updated inspection route; when all inspection tasks are completed, the aircraft will return according to the return route that meets the requirements of remaining battery power and flight safety.
[0130] Finally, the actual captured images, the areas that have been effectively captured, the updated results of the areas that have not been captured, the range of changes in the position of the drainage line, the corresponding applicable time, the corrected shooting position, the execution record of the updated inspection route, and any remaining incomplete status are saved to form the inspection task execution result.
[0131] Through the above steps, the actual shooting results, unshot areas, the range of changes in the location of the diversion line, the estimated arrival time, the corrected shooting position, and the remaining inspection route are continuously updated. When the location of the diversion line continues to change, not only are the shooting task points in the currently unshot areas adjusted, but the subsequent observation path is also uniformly judged based on the expected arrival time of the subsequent shooting task points. After correcting the shooting position or rearranging the route to change the expected arrival time, the range of position changes of the diversion line and the observation path are re-verified based on the new expected arrival time, so as to avoid directly using the corrected shooting position that is only effective at the original expected time to the new execution time.
[0132] Meanwhile, by forming a first route that includes the current offset reshoot and a second route that completes the reshoot using subsequent shooting task points, coverage and flight distance are compared under the same remaining tasks, the same endpoint, and the same actual executable conditions; after executing the updated inspection route, the unshot areas are updated based on the actual captured images rather than the expected coverage, and the position change range of the diversion line is updated using the latest continuous captured images. Therefore, the location and route of the unfinished inspection tasks are continuously adjusted until the necessary shooting requirements are met or the safe stopping conditions are reached.
[0133] The above simulated embodiments are only used to illustrate the specific implementation process of the algorithm and are not intended as actual flight test data. The actual technical effect should be verified through actual inspection tests under the same shooting requirements, the same remaining tasks, and the same safety constraints.
[0134] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.
[0135] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0136] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.
[0137] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0138] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for unmanned aerial vehicle (UAV) inspection task planning for power lines, characterized in that, Includes steps; Select shooting task points that meet the preset number of visits, determine the visiting order based on the flight distance, and generate an initial inspection route; detect whether there are unshot areas on the parts to be inspected based on the images captured by the current shooting task points, and determine the range of position changes of the diversion line if there are unshot areas. Based on the range of changes in the location of the un-photographed area and the diversion line, determine whether the observation path of the current and subsequent photographing task points is blocked by the diversion line. Adjust the position of the affected photographing task points to obtain the corrected photographing position, and form the corrected subsequent inspection route according to the original visit order. Based on the corrected shooting position, a first route is generated after the current shooting task point is offset and reshot, and a second route is generated to supplement the unshot areas and rearrange the unfinished tasks using the corrected subsequent shooting task points. The coverage of the unshot areas by the two routes and the increase in flight distance relative to the corrected subsequent inspection route are compared to determine the updated inspection route. The inspection route is updated, and the location changes of un-photographed areas and diversion lines are updated. If un-photographed areas still exist, the location adjustment and route determination are repeated.
2. The method for planning UAV inspection tasks for power lines according to claim 1, characterized in that, Selecting shooting task points that meet the preset number of visits includes: forming a set task from multiple candidate shooting task points corresponding to the same component to be inspected; selecting shooting task points that meet the preset number of visits from the set task; determining the access order based on the flight distance between shooting task points; and generating an initial inspection route under the condition of meeting the drone's endurance constraints. Save the original shooting area and shooting requirements corresponding to each shooting task point, and record the areas that have been effectively shot and the incomplete status respectively. Do not assume that the corresponding shooting requirements have been completed simply because the shooting task point has been visited.
3. The method for planning UAV inspection tasks for power lines according to claim 1, characterized in that, The detection of uncaptured areas includes: registering the captured image with the original capture area of the component to be inspected; determining the areas that have been effectively captured based on the capture requirements and image registration errors; removing the effectively captured areas from the original capture area to obtain the uncaptured areas; and dividing the uncaptured areas according to the connectivity of the surface of the component to be inspected and counting the number of them. For surfaces with multiple shooting directions or independent key feature requirements, the corresponding shooting completion status is saved separately, and the fact that some surfaces have been effectively photographed does not replace all necessary shooting requirements.
4. The method for planning UAV inspection tasks for power lines according to claim 1, characterized in that, Determining the range of positional changes of the drainage line includes: transforming the image position of the drainage line to the spatial coordinate system of the component to be inspected based on the acquisition time of continuously captured images, camera attitude, and UAV position; determining the positional change rate and positional change acceleration based on the continuous spatial position; and determining the spatial position determination error based on camera calibration error, UAV positioning error, camera attitude error, and image feature matching error. The error is determined based on the space occupied by the drainage line during continuous shooting, the space that may be traversed between adjacent shooting moments, the shape and spatial position of the drainage line itself, and the range of positional variation of the drainage line is determined by combining the verified upper bound of motion. The uncertainty of the predicted position is determined based on the displacement that may be caused by the spatial position determination error and velocity estimation error within the prediction time, as well as the additional displacement corresponding to the verified upper limit of acceleration. The upper limit of the allowable predicted position uncertainty is determined based on the required clearance for the shooting line of sight, camera imaging error, and spatial measurement accuracy. The prediction time corresponding to when the predicted position uncertainty reaches the upper limit of the allowable predicted position uncertainty is taken as the effective prediction time. The applicable time for the position change range of the drainage line is determined based on the expected arrival time of the subsequent shooting task points. If a reliable upper bound on motion cannot be obtained, shorten the applicable time or reacquire continuous images with effective spatial constraints before the execution of subsequent shooting tasks.
5. The method for planning UAV inspection tasks for power lines according to claim 4, characterized in that, The corrected shooting position includes: determining the estimated arrival time of the subsequent shooting task point based on the current actual position, subsequent inspection route, allowable flight speed and shooting dwell time; and judging whether the observation path is blocked by the guide line based on the position change range of the corresponding estimated arrival time within the applicable time range of the guide line's position change. If the applicable time expires, either reacquire continuously captured images or retain the state that needs to be re-verified. The affected shooting task points are determined based on the minimum spatial distance between the observation path and the guide line and the imaging clearance. Available shooting positions are searched within the space that meets the shooting requirements and flight safety requirements. The distance is adjusted according to the position to determine the corrected shooting position, and other available shooting positions with different coverage relationships or subsequent connection distances are retained. After forming the revised subsequent inspection route according to the original visit order, the estimated arrival time is recalculated. When the recalculated estimated arrival time makes the range of positional changes corresponding to the original corrected shooting position no longer applicable or the observation path is obstructed again, the corrected shooting position and estimated arrival time are re-determined. If a consistent result cannot be reached, reacquire continuously captured images, shorten the planning time range, or retain the state that needs to be re-verified.
6. The method for planning UAV inspection tasks for power lines according to claim 5, characterized in that, The generation of the first and second routes includes: when there is only one unshot area and the corrected shooting position corresponding to the current shooting task point can completely shoot the unshot area, the current corrected shooting position is connected to the corrected subsequent inspection route to form the first route; Select available shooting locations from subsequent shooting task points and candidate shooting task points that have not yet been executed, which can fully capture the unshot areas, rearrange the access order of the unfinished tasks, and form a second route; When the corrected subsequent shooting task point can simultaneously complete the original shooting requirements and the reshooting of the unshot areas, it is visited only once and the shooting completion relationship is recorded separately.
7. The method for planning UAV inspection tasks for power lines according to claim 5, characterized in that, The generation of the first and second routes also includes: when there are multiple unshot areas, or when there is only one unshot area but a single shooting position cannot capture it completely, the unshot areas and corresponding surface parts that can be effectively captured by each available shooting position are determined respectively, and the combination of shooting positions that meets the coverage requirements of the unshot areas is determined according to the coverage union of each available shooting position. The corrected shooting position corresponding to the current shooting task point is combined with the available shooting position that can fill the remaining unshot area to form the first route; the available shooting positions that can jointly cover the unshot area are selected from the subsequent shooting task points and the candidate shooting task points that have not yet been executed, and the access order of the unfinished tasks is rearranged to form the second route. Retain available shooting locations with different coverage relationships, subsequent connection distances, or estimated arrival times for route comparison.
8. A method for planning UAV inspection tasks for power lines according to claim 6, characterized in that, Determining the updated inspection route includes: merging the areas that can be effectively photographed by each shooting position in the first and second routes respectively, and checking the coverage requirements of unphotographed areas, the preset number of visits, the original shooting requirements, flight safety requirements, remaining battery power and return-to-home constraints; Based on the same current actual location, remaining tasks and destination, the flight distances of the first route, the second route and the revised subsequent inspection route are accumulated respectively, and the new flight distance is calculated. When the revised subsequent inspection route cannot complete all the necessary shooting tasks, the original tasks that cannot be completed should be supplemented to form a common executable benchmark. When a common executable baseline cannot be established, the complete flight distances of executable candidate routes are compared under the same remaining mission requirements and endpoints, and the shorter distance of an unexecutable route is not used as the selection criterion. The estimated arrival time is recalculated based on the visit order of the first and second routes, and the applicable time and observation path of the location variation range of the diversion line are re-examined. When route rearrangement makes the corrected shooting position no longer meet the requirements, the corrected shooting position, coverage and route distance are re-determined until the corrected shooting position, expected arrival time and coverage relationship all meet the requirements, or the process of acquiring continuous shooting images and re-verifying is initiated. Among the candidate routes that meet all the necessary conditions, the route with the shorter new flight distance is selected as the updated inspection route.
9. A method for planning unmanned aerial vehicle (UAV) inspection tasks for power lines according to claim 8, characterized in that, The update of the unshot areas and the range of changes in the location of the diversion line includes: determining the areas that have been effectively shot based on the supplementary images actually obtained after executing the updated inspection route, removing the areas that have been effectively shot from the original shooting areas, re-dividing the remaining unshot areas and counting the number, retaining the shooting tasks that have been completed, and not directly using the expected coverage as the actual shooting completion result. The range of positional changes of the drainage line is re-determined based on continuously captured images. The spatial difference between the latest range of positional changes and the previous range of positional changes is compared. The change judgment threshold is determined based on spatial measurement error and image registration error. When the spatial difference exceeds the change judgment threshold, the shooting task points that have not yet been executed are re-judged using the latest position change range; If the change judgment threshold is not exceeded, the current route is maintained and sampling continues; for position changes that exceed the applicable time range, they are not used for deterministic occlusion judgment.
10. A method for planning unmanned aerial vehicle (UAV) inspection tasks for power lines according to claim 9, characterized in that, Repeated position adjustment and route determination include: using the current actual position of the drone as the starting point of the subsequent route, regenerating the first and second routes based on the updated uncaptured areas and the coverage relationship of each available capture location, and recalculating the new flight distance using the latest corrected subsequent inspection route as a common benchmark. Route segments that have already been actually flown are no longer included in the current remaining route. When the range of positional variation of the drainage line is no longer applicable to the estimated arrival time of the subsequent shooting task points, reacquire continuous shooting images and determine the corrected shooting position; If an image that meets the shooting requirements still cannot be obtained after multiple position corrections, the system determines whether to continue shooting based on the preset allowed reshoot time, number of reshoots, and remaining battery power. When there is no route that meets the shooting requirements and flight safety requirements, the remaining battery power is insufficient to support reshooting and safe return, or continuous reshooting has not produced a new effective shooting area and the allowed reshooting time has been reached, or there are no other available shooting locations, stop the current reshooting attempt, retain the incomplete state of the unshot area and perform safety procedures. When all un-photographed areas are eliminated and each set of tasks meets the preset access quantity and original shooting requirements, the inspection task of the corresponding component to be inspected is determined to be completed.
