Multi-UAV trajectory planning method and device applicable to large-area power transmission lines

Abstract

The present application relates to a multi-UAV trajectory planning method and device applicable to large-area power transmission lines, the method comprising obtaining inspection area information and inspection task information; constructing a transmission line topology map of the area to be inspected based on the inspection area information and the inspection task information, dividing the transmission line topology map into inspection route units and assigning UAVs based on the path optimization model; planning the first flight trajectory of each UAV based on the transmission tower coordinates, cable length and parking point coordinates in a single inspection route unit; judging the type of the current inspection target based on the image collected by the UAV and executing the corresponding inspection strategy respectively. By dividing the transmission line into several inspection route units, the method enables multiple UAVs to perform inspections and take photos in parallel, which can efficiently and quickly complete their respective inspection tasks, save inspection time, and effectively improve inspection efficiency. At the same time, the flight trajectory is corrected for different inspection route units in a targeted manner to achieve refined inspections and improve the quality of line inspections.

Description

Technical Field

[0001] The present application relates to the field of UAV power inspection technology, and specifically to a multi-UAV trajectory planning method and device suitable for large-area power transmission lines. Background Art

[0002] With the rapid development of power grid infrastructure construction, the distribution range of transmission lines has gradually expanded and the distribution density has gradually increased. Since the transmission line equipment is exposed to the outdoor environment for a long time, natural disasters, animal and plant invasions, and human external damage have caused different degrees of damage to it. In order to maintain the normal operation of the transmission line, it is necessary to regularly patrol and inspect the transmission towers and transmission lines. At present, the use of drone inspections has gradually become a normal means of power inspection.

[0003] The main method of using drones for inspection is to control the drones or a single drone for autonomous inspection. However, due to the wide distribution of power transmission lines, the inspection area is large, and the drone's endurance is limited. It is impossible to complete the inspection task of a large number of long-distance transmission lines alone. The inspection capacity is greatly restricted and the inspection efficiency is low. At the same time, when multiple drones are used for collaborative inspection, due to the complex control logic of multiple drones, it is impossible to achieve accurate inspection and the inspection effect is greatly reduced. Summary of the invention

[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present application provides a multi-UAV trajectory planning method and device applicable to large-area power transmission lines, which specifically adopts the following technical solutions:

[0005] A multi-UAV trajectory planning method applicable to large-area power transmission lines comprises the following steps:

[0006] Obtaining inspection area information and inspection task information; the inspection area information at least includes the inspection area range, tower coordinate information and cable direction information, and the inspection task information at least includes the inspection target type and parking point coordinates;

[0007] A transmission line topology map of the area to be inspected is constructed according to the inspection area information and the inspection task information, and a plurality of inspection route units are obtained by dividing the transmission line topology map based on the path optimization model, and a drone is assigned to each inspection route unit; the inspection route unit includes at least the number of transmission towers, the coordinates of the transmission towers, and the length of the cables;

[0008] Planning the first flight trajectory of each drone according to the transmission tower coordinates, cable length, and parking point coordinates in a single inspection route unit;

[0009] Determine the current inspection target type based on the image collected by the drone, and execute the corresponding inspection strategy according to the inspection target type; the inspection target type includes at least transmission cables and transmission towers;

[0010] When the inspection target type is determined to be a transmission cable, the first inspection strategy is executed:

[0011] In the j The infrared image and radar point cloud data of the inspection target are obtained during the acquisition time;

[0012] Based on the multi-sensor data fusion algorithm, the radar point cloud data of the inspection target at the same acquisition time is fused with the infrared image to obtain the infrared image with radar data;

[0013] Based on the target recognition algorithm, each power transmission cable in the infrared image with radar data is framed and the radar point cloud data within the framed area is selected;

[0014] Acquire the trajectory coordinates of the corresponding power transmission cable according to the radar point cloud data located in the framed area, and correct the first flight trajectory based on the trajectory coordinates of the power transmission cable to obtain the second flight trajectory;

[0015] When the inspection target type is determined to be a transmission tower, the second inspection strategy is executed:

[0016] A preset detection object set is obtained based on the type of transmission tower, and a third flight trajectory is constructed with a reference point or a reference axis of the transmission tower.

[0017] Optional: When obtaining the inspection area information and the inspection task information, the original image of the area to be inspected is obtained through the geographic information system or aerial images, and the positions of the transmission towers, transmission cables and parking points in the area to be inspected are obtained through the target recognition algorithm; then, the transmission towers are used as nodes and the transmission cables are used as node connections to construct a transmission line topology map of the area to be inspected, wherein each node of the transmission line topology map contains at least the coordinates and type of the corresponding transmission tower, and each node connection of the transmission line topology map contains at least the direction and length of the corresponding transmission cable.

[0018] Optionally, the step of dividing the transmission line topology map based on the path optimization model to obtain a plurality of inspection route units includes:

[0019] Taking the shortest inspection time and the least number of involved drones as the optimization goal, a path optimization model is constructed according to the tower coordinate information, cable direction information and parking point coordinates. The objective function of the path optimization model is:

[0020] ;

[0021] in is the minimum inspection time after the kth iteration; is the longest inspection time of all inspection route units after the kth iteration; is the inspection time of the Mth inspection route unit after the kth iteration; is the minimum inspection time of a single inspection route unit after the i-th iteration at the k-th iteration; is the longest inspection time of all drones in a single inspection route unit after the i-th iteration at the k-th iteration; is the inspection time of the nth UAV of a single inspection route unit after the i-th iteration at the k-th iteration; The time it takes for the nth drone in a single inspection route unit to conduct line inspection; The flight time between the inspection starting point and the parking point when the nth UAV in a single inspection route unit conducts line inspection; The flight time between the inspection endpoint and the parking point when the nth UAV in a single inspection route unit conducts line inspection;

[0022] The transmission line topology map of the area to be inspected is input into the path optimization model to obtain a number of inspection route units.

[0023] Optionally: the path optimization model includes a path constraint function and a distance constraint function;

[0024] The path constraint function is used to constrain the path of the inspection route unit after each iteration, and the path constraint function at least includes:

[0025] The first constraint is used to constrain that the nodes of each inspection route unit do not overlap with the nodes of other inspection route units;

[0026] The second constraint is used to constrain the sum of the distances between the starting point and the end point of the route in all inspection route units and the parking point to be the smallest;

[0027] The distance constraint function is used to constrain the maximum flight time of each drone in the inspection route unit:

[0028] ;

[0029] in Allowance for the flight time of the drone; It is the theoretical maximum flight time of the drone.

[0030] Optionally: the step of planning the first flight trajectory of each UAV according to the transmission tower coordinates, cable length and parking point coordinates in a single inspection route unit includes:

[0031] Based on the inspection route units obtained by the path optimization model, the inspection starting points and inspection end points of different drones in each inspection route unit are analyzed;

[0032] Based on the coordinates of the inspection starting point and the inspection end point of the drone, the theoretical inspection trajectory of the drone is calculated:

[0033] ;

[0034] in It is the direction phasor from the inspection starting point to the inspection end point of the drone; They are the three-dimensional coordinates of the inspection starting point of the UAV; They are the three-dimensional coordinates of the inspection endpoints of the UAV; is the trajectory position coefficient, where ,when =0, indicating the inspection starting point coordinates of the corresponding drone. =1, indicating the inspection endpoint coordinates of the corresponding drone; The first The three-dimensional coordinates of the theoretical inspection position when the position is;

[0035] Based on the coordinates of the inspection starting point of the drone and the parking point, the departure trajectory of the drone is calculated;

[0036] Based on the coordinates of the inspection endpoint and the parking point of the drone, the return trajectory of the drone is calculated;

[0037] The first flight trajectory of the drone is generated by combining the theoretical inspection trajectory of the drone, the departure trajectory of the drone and the return trajectory of the drone.

[0038] Optionally, the step of acquiring the trajectory coordinates of the corresponding transmission cable according to the radar point cloud data located in the framed area includes:

[0039] Get the range coordinates of the selected area on the infrared image with radar data;

[0040] Analyze the radar point cloud data contained in the selected area, and perform denoising on the radar point cloud data based on the density-based clustering algorithm;

[0041] A fitting trajectory is obtained by performing curve fitting based on the denoised radar point cloud data, and the trajectory coordinates of the corresponding power transmission cable in the current infrared image are obtained according to the fitting trajectory.

[0042] Optionally: the step of correcting the first flight trajectory based on the trajectory coordinates of the transmission cable includes:

[0043] Based on the trajectory coordinates of the power transmission cable obtained in the infrared image, the distance between the UAV and the power transmission cable at the current acquisition moment is obtained;

[0044] Determine whether the distance between the drone and the power transmission cable at the current collection time meets the preset distance range:

[0045] If the distance between the drone and the power cable at the current collection moment is higher than the preset upper distance limit, the drone is controlled to approach the power cable; if the distance between the drone and the power cable at the current collection moment is lower than the preset lower distance limit, the drone is controlled to move away from the power cable with its back facing away; if the distance between the drone and the power cable at the current collection moment is within the preset distance range, the drone is controlled to maintain the current altitude;

[0046] Based on the theoretical inspection trajectory of the UAV, the distance between the UAV and the transmission cable is controlled within a preset distance range at different collection times, and then the theoretical inspection trajectory of the UAV is corrected to obtain the second flight trajectory.

[0047] Optional: When there are multiple transmission cables in the infrared image, the distance between the UAV and different transmission cables at the current acquisition moment will be determined respectively, and the average value of the distance between the UAV and different transmission cables will be obtained, and then it will be determined whether the corresponding average value meets the preset distance range.

[0048] Optionally, the step of acquiring a preset set of detection objects based on the type of transmission tower and constructing a third flight trajectory with a reference point or a reference axis of the transmission tower includes:

[0049] Using different types of transmission towers to train the neural network model, so as to obtain a neural network model for identifying the type and structural composition of the transmission tower;

[0050] Input the transmission tower image collected by the drone into the trained neural network model, and output the type of transmission tower in the image and the corresponding structural composition; the structural composition at least includes the tower size, ancillary facilities and connecting lines;

[0051] The main structure of the transmission tower is obtained according to the type of transmission tower and the corresponding structural composition analysis, and the corresponding reference point or reference axis is determined by the main structure of the transmission tower; the reference point or reference axis is used to control the UAV to fly around itself according to a preset trajectory;

[0052] The drone is controlled to fly around the transmission tower according to the determined reference point or reference axis, and image data of the corresponding structural components are collected.

[0053] In addition, the present application also discloses a multi-UAV trajectory planning device applicable to large-area power transmission lines, the device comprising:

[0054] The inspection information acquisition module is used to obtain the inspection area information and the inspection task information; the inspection area information at least includes the inspection area range, the tower coordinate information and the cable direction information, and the inspection task information at least includes the inspection target type and the parking point coordinates;

[0055] The inspection route allocation module is used to construct a transmission line topology map of the inspection area according to the inspection area information and the inspection task information, divide the transmission line topology map into several inspection route units based on the path optimization model, and allocate a drone to each inspection route unit; the inspection route unit includes at least the number of transmission towers, the coordinates of the transmission towers and the length of the cables;

[0056] The flight trajectory planning module plans the first flight trajectory of each UAV according to the transmission tower coordinates, cable length and parking point coordinates in a single inspection route unit;

[0057] An inspection strategy determination module is used to determine the current inspection target type based on the image collected by the drone, and execute the corresponding inspection strategy according to the inspection target type; the inspection target type includes at least transmission cables and transmission towers;

[0058] When the inspection target type is determined to be a transmission cable, the first inspection strategy is executed:

[0059] In the j The infrared image and radar point cloud data of the inspection target are obtained during the acquisition time;

[0060] Based on the multi-sensor data fusion algorithm, the radar point cloud data of the inspection target at the same acquisition time is fused with the infrared image to obtain the infrared image with radar data;

[0061] Based on the target recognition algorithm, each power transmission cable in the infrared image with radar data is framed and the radar point cloud data within the framed area is selected;

[0062] Acquire the trajectory coordinates of the corresponding power transmission cable according to the radar point cloud data located in the framed area, and correct the first flight trajectory based on the trajectory coordinates of the power transmission cable to obtain the second flight trajectory;

[0063] When the inspection target type is determined to be a transmission tower, the second inspection strategy is executed:

[0064] A preset detection object set is obtained based on the type of transmission tower, and a third flight trajectory is constructed with a reference point or a reference axis of the transmission tower.

[0065] Beneficial Effects

[0066] The technical solution of this application has the following beneficial effects:

[0067] The multi-UAV trajectory planning method of the present application divides the power transmission lines in the inspection area into several inspection route units through a path optimization model, and allocates an appropriate number of UAVs to each inspection route unit, so that multiple UAVs can perform tower inspections and take photos in parallel for their respective inspection route units, thereby achieving efficient and rapid completion of their assigned inspection tasks, greatly reducing manual intervention operations, reducing error rates, saving inspection time, and effectively improving inspection efficiency. At the same time, the flight trajectory can be modified specifically for different inspection route units to achieve refined inspections and improve the quality of line inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 Flowchart of the multi-UAV trajectory planning method in an embodiment of the present application.

[0069] Figure 2 This is a schematic diagram of the power transmission line structure in a certain area to be inspected in an embodiment of the present application.

[0070] Figure 3 This is a topological diagram of a power transmission line in a certain area to be inspected in an embodiment of the present application.

[0071] Figure 4 Schematic diagram of flight trajectory correction in a power transmission cable in an embodiment of the present application.

[0072] Figure 5 This is a structural diagram of a multi-UAV trajectory planning device in an embodiment of the present application.

[0073] Figure 6 This is a structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The present application is further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and cannot be used to limit the scope of protection of the present application. It should be noted that the following detailed descriptions are all exemplary and are intended to provide further explanation of the present application.

[0075] At present, when inspecting large-area transmission lines, most of them use manually controlled drones, that is, the operator remotely controls the drone to fly along the transmission line, where the flight trajectory and flight altitude of the drone are controlled by the operator. This method is greatly affected by the operator's control level, and for large areas, multiple operators are required to conduct inspections, and the inspection cost and inspection efficiency are low. Based on the rapid development of drone collaborative control technology, when inspecting large-area transmission lines, using multiple drones for collaborative control can greatly reduce inspection time and cost and improve inspection efficiency. However, when using collaboratively controlled drones for line inspections, since the collaborative control system is still in point-to-point flight route planning, it is impossible to adaptively change the trajectory of the transmission line, which results in low inspection accuracy and poor inspection quality, and it is impossible to accurately inspect the transmission line.

[0076] Based on the current multi-UAV collaborative control method, this application is optimized according to the point-to-point flight path method, that is, by using infrared images and radar point clouds to obtain the real-time distance between the UAV and the transmission line, while ensuring that the distance between the UAV and the transmission line is fixed, the point-to-point flight path method is optimized and adjusted to form an optimized flight path based on the actual direction of the transmission line, thereby ensuring that inspection images with consistent image quality are obtained and ensuring accurate inspection results.

[0077] Combination Figure 1 As shown, the embodiment of the present application discloses a multi-UAV trajectory planning method applicable to large-area power transmission lines, which includes the following steps:

[0078] Step S1: Obtain inspection area information and inspection task information; the inspection area information at least includes the inspection area range, tower coordinate information and cable direction information, and the inspection task information at least includes the inspection target type and parking point coordinates.

[0079] It should be noted that when obtaining the inspection area information and the inspection task information in step S1 of the present application, the original image of the area to be inspected can be obtained through the geographic information system or aerial images, and the positions of the transmission towers, transmission cables and parking points in the area to be inspected can be obtained through the existing target recognition algorithm; then, the transmission towers are used as nodes and the transmission cables are used as node connections to construct a transmission line topology map of the area to be inspected, wherein each node of the transmission line topology map at least includes the coordinates and type of the corresponding transmission tower, and each node connection of the transmission line topology map at least includes the direction and length of the corresponding transmission cable. It should be further explained that the parking point in this application generally refers to the take-off position of the drone. At present, there are generally fixed machine nests and mobile machine nests for the large-scale drone inspection stage. Fixed machine nests refer to fixed drone charging warehouses set up at one or more locations in a certain area. When the drone performs an inspection task, it takes off from the fixed drone charging warehouse and flies to the task point for inspection; while the mobile machine nest is generally a mobile vehicle equipment with an integrated drone charging warehouse. When performing an inspection task, the mobile vehicle equipment can be driven to a specific location in the area to be inspected (generally the center of the area to be inspected), and this specific location is used as the take-off point of the drone, and the drone is controlled to fly to the corresponding task point for inspection. This application can perform trajectory planning for both types of parking points.

[0080] Step S2: construct a transmission line topology map of the area to be inspected based on the inspection area information and the inspection task information, divide the transmission line topology map into several inspection route units based on the path optimization model, and assign a drone to each inspection route unit; the inspection route unit includes at least the number of transmission towers, the coordinates of the transmission towers, and the length of the cables;

[0081] Step S3: planning the first flight trajectory of each UAV according to the transmission tower coordinates, cable length and parking point coordinates in a single inspection route unit;

[0082] Step S4: Determine the current inspection target type based on the image collected by the drone, and execute the corresponding inspection strategy according to the inspection target type; the inspection target type includes at least transmission cables and transmission towers. It should be understood that different inspection strategies need to be executed according to the types of transmission cables and transmission towers in the transmission line. The reason is that the target volume of the transmission tower is large, and the collected image is easy to be identified. According to the structure and volume of the transmission tower, the drone generally needs to fly around the transmission tower and shoot specific parts and structures of the transmission tower. However, due to the thin diameter of the transmission cable, it is not easy to be identified in the collected image. At this time, it is necessary to combine infrared and radar point clouds for accurate identification; and the tension of the transmission cable is different due to the temperature difference in different seasons. The position of the transmission cable between adjacent transmission towers is different in different seasons. At this time, the drone needs to have the ability to identify the cable position with high precision to ensure the quality of the image taken during the subsequent inspection.

[0083] Step S41: When it is determined that the inspection target type is a transmission cable, the first inspection strategy is executed:

[0084] In the j The infrared image and radar point cloud data of the inspection target are obtained during the acquisition time;

[0085] Based on the multi-sensor data fusion algorithm, the radar point cloud data of the inspection target at the same acquisition time is fused with the infrared image to obtain the infrared image with radar data;

[0086] Based on the target recognition algorithm, each power transmission cable in the infrared image with radar data is framed and the radar point cloud data within the framed area is selected;

[0087] Acquire the trajectory coordinates of the corresponding power transmission cable according to the radar point cloud data located in the framed area, and correct the first flight trajectory based on the trajectory coordinates of the power transmission cable to obtain the second flight trajectory;

[0088] Step S42: When it is determined that the inspection target type is a transmission tower, the second inspection strategy is executed:

[0089] A preset detection object set is obtained based on the type of transmission tower, and a third flight trajectory is constructed with a reference point or a reference axis of the transmission tower.

[0090] Specifically, in step S2 of the present application, the transmission line topology map is divided into several inspection route units based on the path optimization model. By iterating the transmission line topology map multiple times through the path optimization model, the inspection route with the shortest time consumption and the least number of drones involved can be planned to reduce the inspection cost and improve the inspection efficiency. The specific method of step S2 includes:

[0091] Firstly, the optimization goal is to minimize the inspection time and involve the least number of drones. A path optimization model is constructed based on the tower coordinate information, cable direction information, and parking point coordinates. The objective function of the path optimization model is:

[0092] ;

[0093] in is the minimum inspection time after the kth iteration; is the longest inspection time of all inspection route units after the kth iteration; is the inspection time of the Mth inspection route unit after the kth iteration; is the minimum inspection time of a single inspection route unit after the i-th iteration at the k-th iteration; is the longest inspection time of all drones in a single inspection route unit after the i-th iteration at the k-th iteration; is the inspection time of the nth UAV of a single inspection route unit after the i-th iteration at the k-th iteration; The time it takes for the nth drone in a single inspection route unit to conduct line inspection; The flight time between the inspection starting point and the parking point when the nth UAV in a single inspection route unit conducts line inspection; The flight time between the inspection endpoint and the parking point when the nth UAV in a single inspection route unit conducts line inspection;

[0094] Finally, the transmission line topology map of the area to be inspected is input into the path optimization model to obtain several inspection route units.

[0095] Furthermore, for the path optimization model in step S2, path constraints and distance constraints are also required to ensure that the inspection routes do not conflict with each other, while also meeting the requirements of short overall flight distance and energy consumption of the drone. Specifically, the path optimization model includes a path constraint function and a distance constraint function;

[0096] The path constraint function is used to constrain the path of the inspection route unit after each iteration, which at least includes:

[0097] The first constraint is used to constrain the nodes of each inspection route unit to not be repeated with the nodes of other inspection route units. Figure 2 and Figure 3 As shown in FIG, the path optimization model divides a plurality of inspection route units based on the transmission line topology map of the area to be inspected, such as Figure 3As shown in the figure, nodes 1-7 represent transmission towers, node lines Ⅰ-VI represent transmission cables between two transmission towers, and three inspection route units are obtained through the path optimization model: inspection route 1 (1-Ⅰ-2-Ⅱ-3-Ⅲ-4), inspection route 2 (Ⅳ-5-Ⅴ-6) and inspection route 3 (Ⅵ-7). The nodes and node lines in the three inspection routes are not repeated, which can avoid repeated inspections of a transmission tower or transmission line by drones, resulting in increased inspection time costs.

[0098] The second constraint is used to constrain the sum of the distances between the starting point and the end point of all inspection route units and the parking point to be the minimum. It should be noted that the present application can ensure that the distance between each inspection route unit generated by the path optimization model and the parking point is the minimum through the second constraint, thereby ensuring that the energy consumption of the drone is maximized during the line inspection stage, reducing the round-trip energy consumption of the drone, and thus improving the effective inspection efficiency of the drone. It should be understood that the line starting point or line end point in the present application refers to the path starting point or end point of a single inspection route unit, and the inspection starting point and inspection end point refer to the flight starting point or flight end point when the drone performs an inspection operation.

[0099] The distance constraint function is used to constrain the maximum flight time of each drone in the inspection route unit:

[0100] ;

[0101] in The time it takes for the nth drone in a single inspection route unit to conduct line inspection; The flight time between the inspection starting point and the parking point when the nth UAV in a single inspection route unit conducts line inspection; The flight time between the inspection endpoint and the parking point when the nth UAV in a single inspection route unit conducts line inspection; Allowance for the flight time of the drone; is the theoretical maximum flight time of the drone. It should be noted that the distance constraint function can ensure that the generated inspection route unit and the number of assigned drones meet the energy consumption requirements of a single drone, so as to avoid the risk of crashing due to the exhaustion of energy during the drone inspection. In addition, the time taken by a single drone to conduct line inspection in each inspection route unit includes the time taken by the drone to inspect the transmission tower and the time taken by the drone to inspect the transmission cable. When the path optimization model performs route planning, the drone flight speed is generally set to a fixed speed, or when the drone travels between the inspection location and the parking point, it flies at a specific cruising speed; when the drone conducts an inspection, it flies at a preset inspection speed.

[0102] More specifically, in step S3 of the present application, the first flight trajectory of each drone can be planned and obtained according to the transmission tower coordinates, cable length and parking point coordinates in a single inspection route unit, and the specific planning method includes:

[0103] First, based on the inspection route unit obtained by the path optimization model, the inspection starting point and inspection end point of different drones in each inspection route unit are analyzed; for example, two drones are assigned to the inspection route 1 (1-Ⅰ-2-Ⅱ-3-Ⅲ-4), of which the first drone performs inspection flight along the inspection route 1-Ⅰ-2-Ⅱ, at which time its corresponding inspection starting point is node 1, and the inspection end point is the end of the node connection Ⅱ; while the second drone performs inspection flight along the inspection route 3-Ⅲ-4, at which time its corresponding inspection starting point is node 3, and the inspection end point is node 4. Furthermore, when the inspection end point of the current drone is located at a certain position of a certain node connection line, the inspection starting point of the next drone is the inspection end point of the previous drone to ensure the continuity of the inspection process.

[0104] Then, based on the coordinates of the inspection starting point and the inspection end point of the drone, the theoretical inspection trajectory of the drone is calculated:

[0105] ;

[0106] in It is the direction phasor from the inspection starting point to the inspection end point of the drone; They are the three-dimensional coordinates of the inspection starting point of the UAV; They are the three-dimensional coordinates of the inspection endpoints of the UAV; is the trajectory position coefficient, where ,when =0, indicating the inspection starting point coordinates of the corresponding drone. =1, indicating the inspection endpoint coordinates of the corresponding drone; The first The three-dimensional coordinates of the theoretical inspection position at the time of position; it should be understood that the theoretical inspection trajectory is based on the three-dimensional space connection of adjacent nodes. For example, the theoretical inspection trajectory of inspection route Ⅵ-7 is a straight line connection trajectory obtained by calculating the three-dimensional coordinates of node 5 and node 7 (a straight line connection in three-dimensional space, not a two-dimensional line segment in the transmission line topology diagram).

[0107] Then, based on the coordinates of the inspection starting point of the drone and the parking point, the departure trajectory of the drone is calculated;

[0108] Based on the coordinates of the inspection endpoint and the parking point of the drone, the return trajectory of the drone is calculated;

[0109] Finally, the first flight trajectory of the drone is generated by combining the theoretical inspection trajectory of the drone, the departure trajectory of the drone and the return trajectory of the drone.

[0110] It should be understood that the first flight trajectory generated in this application is mainly used as a reference trajectory line for the UAV flight. In addition, the first flight trajectory is also conducive to the path optimization model to plan the inspection route unit and allocate the UAVs respectively. When the UAV actually flies, the first flight trajectory is used as a reference, and combined with its own obstacle avoidance equipment, the UAV can achieve safe and stable flight.

[0111] Furthermore, in the step S41, the present application uses a combination of infrared images and laser point clouds to accurately locate the position of the transmission cable, thereby accurately determining the relative position of the transmission cable and the drone, and ensuring that the drone always maintains a fixed distance from the transmission cable during flight.

[0112] Specifically, in step S41, the radar point cloud data of the inspection target at the same acquisition time is first fused with the infrared image based on the existing multi-sensor data fusion algorithm to obtain an infrared image with radar data. Generally, the fusion of radar point cloud data and infrared image can adopt a fusion method based on feature extraction, a method based on image fusion, or a fusion method based on deep learning, which are all existing technologies and are not described in detail here.

[0113] In more detail, after the radar point cloud data is fused with the infrared image and an infrared image with radar data is obtained, the present application distinguishes the transmission cables by performing target identification and selection on the infrared image. Since infrared images can reflect the thermal radiation characteristics of the target, they have better detection capabilities for targets with obvious temperature differences. Transmission cables generally bear high voltage currents, and the cables have a higher temperature than the surrounding environment. The image boundary between the transmission cables and the surrounding environment can be clearly distinguished through infrared images. Furthermore, the present application obtains the trajectory coordinates of the corresponding transmission cable based on the radar point cloud data located in the selected area, and the specific steps include:

[0114] First, obtain the range coordinates of the selected area on the infrared image with radar data;

[0115] Then, the radar point cloud data contained in the selected area is analyzed, and the radar point cloud data is denoised based on the density-based clustering algorithm; this step can remove abnormal data points in the selected area to improve the accuracy of subsequent fitting results.

[0116] After that, curve fitting is performed based on the denoised radar point cloud data to obtain a fitting trajectory, and the trajectory coordinates of the corresponding power transmission cable in the current infrared image are obtained according to the fitting trajectory. It should be further explained that when there are multiple power transmission cables in the infrared image, the infrared image has multiple framed areas. At this time, denoising and fitting can be performed for each framed area, and then the distance between the drone and different power transmission cables at the current acquisition moment is determined respectively, and the average value of the distance between the drone and different power transmission cables is obtained. Furthermore, although the frame selection is performed on a two-dimensional infrared image in the present application, the radar point cloud data in the framed area is actually three-dimensional coordinate data, so the fitting trajectory obtained by curve fitting of the radar point cloud data is still a trajectory line in three-dimensional space.

[0117] Further, such as Figure 4 As shown, in step S41 of the present application, the first flight trajectory A can be corrected based on the trajectory coordinates of the transmission cable obtained, and the specific steps include:

[0118] Firstly, the distance between the UAV and the power transmission cable at the current acquisition moment is obtained based on the trajectory coordinates of the power transmission cable obtained in the infrared image;

[0119] Determine whether the distance between the drone and the power transmission cable at the current collection time meets the preset distance range:

[0120] If the distance between the drone and the transmission cable at the current collection moment is higher than the preset upper limit, the drone is controlled to approach the transmission cable; if the distance between the drone and the transmission cable at the current collection moment is lower than the preset lower limit, the drone is controlled to move away from the transmission cable with its back facing away; if the distance between the drone and the transmission cable at the current collection moment is within the preset distance range, the drone is controlled to maintain the current height; for the drone's simultaneous inspection of multiple transmission lines, in order to ensure the safety of the drone and the quality of the inspection and shooting, the distance between the drone and the transmission cable generally needs to meet a certain range of conditions, that is, the preset upper limit and the preset lower limit are set for the distance between the drone and the transmission cable, and the distance between the drone and the transmission cable is within the preset upper limit and the preset lower limit. If the average value is within the preset upper limit and the preset lower limit, the average value is used as the distance between the drone and the transmission cable; when the average value is lower than the preset lower limit, the preset lower limit is used as the distance between the drone and the transmission cable; when the average value is higher than the preset upper limit, the preset upper limit is used as the distance between the drone and the transmission cable.

[0121] Taking the theoretical inspection trajectory of the UAV (i.e., the first flight trajectory A) as a benchmark, the distance between the UAV and the transmission cable is controlled within a preset distance range at different acquisition times, and then the theoretical inspection trajectory of the UAV is corrected to obtain the second flight trajectory B.

[0122] More specifically, in step S42 of the present application, a preset set of detection objects is obtained based on the type of transmission tower, and a third flight trajectory is constructed with a reference point or a reference axis of the transmission tower. The specific steps include:

[0123] Firstly, different types of transmission towers are used to train the neural network model to obtain a neural network model for identifying the types and structural composition of transmission towers. Currently, the types of transmission towers are generally divided into wine glass type, cat head type, upper character type and dry character type, etc. By using the pre-trained neural network model, the type of transmission tower can be identified, and the corresponding structural composition can be obtained based on the analysis of the transmission tower type.

[0124] Then, the transmission tower image collected by the drone is input into the trained neural network model, and the type of transmission tower and the corresponding structural composition in the output image are output; wherein the structural composition at least includes the tower size, ancillary facilities and connecting lines;

[0125] After that, the main structure of the transmission tower (i.e., the frame support structure of the transmission tower) is obtained according to the type of transmission tower and the corresponding structural composition, and the corresponding reference point or reference axis is determined by the main structure of the transmission tower; generally, the transmission tower is a symmetrical structure as a whole, and the auxiliary equipment is generally installed at the upper position of the tower, so the reference axis can be the symmetrical central axis of the transmission tower, or the reference point can be a certain position at the top of the symmetrical central axis of the transmission tower. The reference point or reference axis is used to control the drone to fly around itself according to the preset trajectory; it should be noted that the preset trajectory of the drone in this step can be automatically generated according to the tower size. For example, after obtaining the tower size, the computer or control center will automatically generate the flight trajectory of the corresponding transmission tower according to the parameters such as the drone's circling spacing and flight speed. At this time, the drone can automatically fly according to the generated flight trajectory combined with the obstacle avoidance equipment.

[0126] The drone is controlled to fly around the transmission tower according to the determined reference point or reference axis, and image data of the corresponding structural components are collected.

[0127] In addition, combined Figure 5 As shown, the present application also discloses a multi-UAV trajectory planning device applicable to large-area power transmission lines, the device comprising:

[0128] The inspection information acquisition module is used to obtain the inspection area information and the inspection task information; the inspection area information at least includes the inspection area range, the tower coordinate information and the cable direction information, and the inspection task information at least includes the inspection target type and the parking point coordinates;

[0129] The inspection route allocation module is used to construct a transmission line topology map of the inspection area according to the inspection area information and the inspection task information, divide the transmission line topology map into several inspection route units based on the path optimization model, and allocate a drone to each inspection route unit; the inspection route unit includes at least the number of transmission towers, the coordinates of the transmission towers and the length of the cables;

[0130] The flight trajectory planning module plans the first flight trajectory of each UAV according to the transmission tower coordinates, cable length and parking point coordinates in a single inspection route unit;

[0131] An inspection strategy determination module is used to determine the current inspection target type based on the image collected by the drone, and execute the corresponding inspection strategy according to the inspection target type; the inspection target type includes at least transmission cables and transmission towers;

[0132] When the inspection target type is determined to be a transmission cable, the first inspection strategy is executed:

[0133] In the j The infrared image and radar point cloud data of the inspection target are obtained during the acquisition time;

[0134] Based on the multi-sensor data fusion algorithm, the radar point cloud data of the inspection target at the same acquisition time is fused with the infrared image to obtain the infrared image with radar data;

[0135] Based on the target recognition algorithm, each power transmission cable in the infrared image with radar data is framed and the radar point cloud data within the framed area is selected;

[0136] Acquire the trajectory coordinates of the corresponding power transmission cable according to the radar point cloud data located in the framed area, and correct the first flight trajectory based on the trajectory coordinates of the power transmission cable to obtain the second flight trajectory;

[0137] When the inspection target type is determined to be a transmission tower, the second inspection strategy is executed:

[0138] A preset detection object set is obtained based on the type of transmission tower, and a third flight trajectory is constructed with a reference point or a reference axis of the transmission tower.

[0139] The device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment will not be described again here.

[0140] like Figure 6 As shown, the embodiment of the present application also provides an electronic device, including a processor and a memory, a program or instruction stored in the memory and executable on the processor, and when the program or instruction is executed by the processor, the following is achieved: Figure 1The various processes of the method embodiment shown in the figure can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0141] The embodiment of the present application also provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the above Figure 1 The various processes of the method embodiments described above can achieve the same technical effect, and will not be described again here to avoid repetition.

[0142] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 1 The various processes of the method embodiments described above can achieve the same technical effect, and will not be described again here to avoid repetition.

[0143] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0144] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0145] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another device, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0146] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0147] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0148] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0149] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device (which can be a terminal or a platform, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0150] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A multi-UAV trajectory planning method applicable to large-area power transmission lines, characterized in that: The steps include: Obtaining inspection area information and inspection task information; the inspection area information at least includes the inspection area range, tower coordinate information and cable direction information, and the inspection task information at least includes the inspection target type and parking point coordinates; A transmission line topology map of the area to be inspected is constructed based on the inspection area information and inspection task information. The transmission line topology map is divided into several inspection route units based on the path optimization model, and a drone is assigned to each inspection route unit. The inspection route unit includes at least the number of transmission towers, the coordinates of the transmission towers and the length of the cables; Plan the first flight trajectory of each drone according to the transmission tower coordinates, cable length, and parking point coordinates in a single inspection route unit; Determine the type of the current inspection target based on the images collected by the drone, and execute the corresponding inspection strategy according to the inspection target type; The inspection target types include at least transmission cables and transmission towers; When the inspection target type is determined to be a transmission cable, the first inspection strategy is executed: In the j The infrared image and radar point cloud data of the inspection target are obtained during the acquisition time; Based on the multi-sensor data fusion algorithm, the radar point cloud data of the inspection target at the same acquisition time is fused with the infrared image to obtain the infrared image with radar data; Based on the target recognition algorithm, each power transmission cable in the infrared image with radar data is framed and the radar point cloud data within the framed area is selected; Acquire the trajectory coordinates of the corresponding power transmission cable according to the radar point cloud data located in the framed area, and correct the first flight trajectory based on the trajectory coordinates of the power transmission cable to obtain the second flight trajectory; When the inspection target type is determined to be a transmission tower, the second inspection strategy is executed: A preset detection object set is obtained based on the type of transmission tower, and a third flight trajectory is constructed with a reference point or a reference axis of the transmission tower.

2. The multi-UAV trajectory planning method according to claim 1, characterized in that: When obtaining the inspection area information and the inspection task information, the original image of the area to be inspected is obtained through the geographic information system or the aerial image, and the positions of the transmission towers, transmission cables and parking points in the area to be inspected are obtained through the target recognition algorithm; then, the transmission towers are used as nodes and the transmission cables are used as node connections to construct a transmission line topology map of the area to be inspected, wherein each node of the transmission line topology map at least includes the coordinates and type of the corresponding transmission tower, and each node connection of the transmission line topology map at least includes the direction and length of the corresponding transmission cable.

3. The multi-UAV trajectory planning method according to claim 2, characterized in that: The step of dividing the transmission line topology map based on the path optimization model to obtain a plurality of inspection route units comprises: Taking the shortest inspection time and the least number of involved drones as the optimization goal, a path optimization model is constructed according to the tower coordinate information, cable direction information and parking point coordinates. The objective function of the path optimization model is: ； in is the minimum inspection time after the kth iteration; is the longest inspection time of all inspection route units after the kth iteration; is the inspection time of the Mth inspection route unit after the kth iteration; is the minimum inspection time of a single inspection route unit after the i-th iteration at the k-th iteration; is the longest inspection time of all drones in a single inspection route unit after the i-th iteration at the k-th iteration; is the inspection time of the nth UAV of a single inspection route unit after the i-th iteration at the k-th iteration; The time it takes for the nth drone in a single inspection route unit to conduct line inspection; The flight time between the inspection starting point and the parking point when the nth UAV in a single inspection route unit conducts line inspection; The flight time between the inspection endpoint and the parking point when the nth UAV in a single inspection route unit conducts line inspection; The transmission line topology map of the area to be inspected is input into the path optimization model to obtain a number of inspection route units.

4. The multi-UAV trajectory planning method according to claim 3, characterized in that: The path optimization model includes a path constraint function and a distance constraint function; The path constraint function is used to constrain the path of the inspection route unit after each iteration, and the path constraint function at least includes: The first constraint is used to constrain that the nodes of each inspection route unit do not overlap with the nodes of other inspection route units; The second constraint is used to constrain the sum of the distances between the starting point and the end point of the route in all inspection route units and the parking point to be the smallest; The distance constraint function is used to constrain the maximum flight time of each drone in the inspection route unit: ； in Allowance for the flight time of the drone; It is the theoretical maximum flight time of the drone.

5. The multi-UAV trajectory planning method according to claim 1, characterized in that: The step of planning the first flight trajectory of each UAV according to the transmission tower coordinates, cable length and parking point coordinates in a single inspection route unit includes: Based on the inspection route units obtained by the path optimization model, the inspection starting points and inspection end points of different drones in each inspection route unit are analyzed; Based on the coordinates of the inspection starting point and the inspection end point of the drone, the theoretical inspection trajectory of the drone is calculated: ； in It is the direction phasor from the inspection starting point to the inspection end point of the drone; They are the three-dimensional coordinates of the inspection starting point of the UAV; They are the three-dimensional coordinates of the inspection endpoints of the UAV; is the trajectory position coefficient, where ,when =0, indicating the inspection starting point coordinates of the corresponding drone. =1, indicating the inspection endpoint coordinates of the corresponding drone; The first The three-dimensional coordinates of the theoretical inspection position when the position is; Based on the coordinates of the inspection starting point of the drone and the parking point, the departure trajectory of the drone is calculated; Based on the coordinates of the inspection endpoint and the parking point of the drone, the return trajectory of the drone is calculated; The first flight trajectory of the drone is generated by combining the theoretical inspection trajectory of the drone, the departure trajectory of the drone and the return trajectory of the drone.

6. The multi-UAV trajectory planning method according to claim 1, characterized in that: The step of obtaining the trajectory coordinates of the corresponding transmission cable according to the radar point cloud data located in the framed area includes: Get the range coordinates of the selected area on the infrared image with radar data; Analyze the radar point cloud data contained in the selected area, and perform denoising on the radar point cloud data based on the density-based clustering algorithm; A fitting trajectory is obtained by performing curve fitting based on the denoised radar point cloud data, and the trajectory coordinates of the corresponding power transmission cable in the current infrared image are obtained according to the fitting trajectory.

7. The multi-UAV trajectory planning method according to claim 6, characterized in that: The step of correcting the first flight trajectory based on the trajectory coordinates of the power transmission cable includes: Based on the trajectory coordinates of the power transmission cable obtained in the infrared image, the distance between the UAV and the power transmission cable at the current acquisition moment is obtained; Determine whether the distance between the drone and the power transmission cable at the current collection time meets the preset distance range: If the distance between the drone and the power cable at the current collection moment is higher than the preset upper distance limit, the drone is controlled to approach the power cable; if the distance between the drone and the power cable at the current collection moment is lower than the preset lower distance limit, the drone is controlled to move away from the power cable with its back facing away; if the distance between the drone and the power cable at the current collection moment is within the preset distance range, the drone is controlled to maintain the current altitude; Based on the theoretical inspection trajectory of the UAV, the distance between the UAV and the transmission cable is controlled within a preset distance range at different collection times, and then the theoretical inspection trajectory of the UAV is corrected to obtain the second flight trajectory.

8. The multi-UAV trajectory planning method according to claim 7, characterized in that: When there are multiple transmission cables in the infrared image, the distance between the UAV and different transmission cables at the current acquisition time will be determined respectively, and the average value of the distance between the UAV and different transmission cables will be obtained, and then it will be determined whether the corresponding average value meets the preset distance range.

9. The multi-UAV trajectory planning method according to claim 1, characterized in that: The step of acquiring a preset detection object set based on the type of the transmission tower and constructing a third flight trajectory with a reference point or a reference axis of the transmission tower comprises: Using different types of transmission towers to train the neural network model, so as to obtain a neural network model for identifying the type and structural composition of the transmission tower; Input the transmission tower image collected by the drone into the trained neural network model, and output the type of transmission tower in the image and the corresponding structural composition; the structural composition at least includes the tower size, ancillary facilities and connecting lines; The main structure of the transmission tower is obtained according to the type of transmission tower and the corresponding structural composition analysis, and the corresponding reference point or reference axis is determined by the main structure of the transmission tower; the reference point or reference axis is used to control the UAV to fly around itself according to a preset trajectory; The drone is controlled to fly around the transmission tower according to the determined reference point or reference axis, and image data of the corresponding structural components are collected.

10. A multi-UAV trajectory planning device suitable for large-area power transmission lines, characterized in that: The device comprises: The inspection information acquisition module is used to obtain the inspection area information and the inspection task information; the inspection area information at least includes the inspection area range, the tower coordinate information and the cable direction information, and the inspection task information at least includes the inspection target type and the parking point coordinates; The inspection route allocation module is used to construct a transmission line topology map of the inspection area according to the inspection area information and the inspection task information, divide the transmission line topology map into several inspection route units based on the path optimization model, and allocate a drone to each inspection route unit; the inspection route unit includes at least the number of transmission towers, the coordinates of the transmission towers and the length of the cables; The flight trajectory planning module plans the first flight trajectory of each UAV according to the transmission tower coordinates, cable length and parking point coordinates in a single inspection route unit; An inspection strategy determination module is used to determine the current inspection target type based on the image collected by the drone, and execute the corresponding inspection strategy according to the inspection target type; the inspection target type includes at least transmission cables and transmission towers; When the inspection target type is determined to be a transmission cable, the first inspection strategy is executed: In the j The infrared image and radar point cloud data of the inspection target are obtained during the acquisition time; Based on the multi-sensor data fusion algorithm, the radar point cloud data of the inspection target at the same acquisition time is fused with the infrared image to obtain the infrared image with radar data; Based on the target recognition algorithm, each power transmission cable in the infrared image with radar data is framed and the radar point cloud data within the framed area is selected; Acquire the trajectory coordinates of the corresponding power transmission cable according to the radar point cloud data located in the framed area, and correct the first flight trajectory based on the trajectory coordinates of the power transmission cable to obtain the second flight trajectory; When the inspection target type is determined to be a transmission tower, the second inspection strategy is executed: A preset detection object set is obtained based on the type of transmission tower, and a third flight trajectory is constructed with a reference point or a reference axis of the transmission tower.

Images