A method for automatically generating waypoints based on substation target detection clustering

Abstract

The application discloses a kind of based on substation target detection clustering cluster waypoint automatic generation method, specifically relates to unmanned aerial vehicle inspection field, including establishing substation three-dimensional diagram, the RGB image of equipment needing to be detected is collected, the geometric center of each equipment on the plane is calculated using Graph-Detect3D, according to equipment type, orientation, the coordinate information of all equipment to be inspected target is obtained according to the geometric center of equipment, all equipment to be inspected target is numbered, the geometric center of equipment is clustered using density clustering algorithm DBSCAN, all equipment center geometric center in the same cluster is used as waypoint, and waypoint is generated according to the geometric center coordinates of each class, so the number of waypoints will not be redundant, will be just covered to each equipment, can collect the picture of all equipment of substation with least waypoint, reduce the redundancy of data collection.

Description

Technical fields:

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) inspection, and in particular to a method for automatically generating waypoints based on target detection clusters in substations. Background technology:

[0002] Drone inspection is an important method for substation inspection, enabling the monitoring of equipment operation and changes in the surrounding environment. It allows for the timely detection of equipment defects and safety hazards, preventing accidents and ensuring the safety of power transmission and transformation equipment and the stability of the power system. However, before drone inspection, waypoint data collection and route planning are necessary. Currently, substation waypoint data collection mainly relies on on-site surveys and manual mapping. This method is subjective, often resulting in incomplete substation area coverage, redundant waypoints, and is extremely time-consuming and labor-intensive.

[0003] Based on this, the present invention aims to provide an automatic waypoint generation method based on substation target detection clusters, which solves the problems of incomplete waypoint coverage of substation areas and waypoint redundancy. Summary of the Invention:

[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide an automatic waypoint generation method based on substation target detection clusters. By automatically generating inspection waypoints through equipment clusters, the method not only generates waypoints efficiently but also avoids redundancy in the number of waypoints. The method obtains the coordinate information of all equipment to be inspected targets based on equipment type, orientation, and geometric center of the equipment, and assigns numbers to all equipment to be inspected targets, which can prevent the problem of uncovered inspection targets.

[0005] The technical solution of the present invention is as follows:

[0006] An automatic waypoint generation method based on substation target detection clusters includes the following steps:

[0007] (1) Use drones to take aerial photos of the substation, create a 3D map of the substation, and mark the location information of each piece of equipment;

[0008] (2) Set the camera focal length, shooting distance and gimbal angle parameters, and use the drone to take pictures of all the equipment that needs to be detected;

[0009] (3) The RGB image collected in step (2) is used as the input of the graph-based 3D target detection deep network Graph-Detect3D to obtain the equipment type, the 3D bounding box of the target to be inspected on the equipment and its orientation in the substation.

[0010] (4) Calculate the geometric center of each device on the plane based on the coordinate information of the vertices of the 3D bounding box of the device in the target detection bird's-eye view output;

[0011] (5) Obtain the coordinate information of all equipment to be inspected based on the equipment type, orientation, and geometric center of the equipment, and number all equipment to be inspected.

[0012] (6) The density clustering algorithm DBSCAN is used to cluster the geometric center of the equipment, and multiple clusters are obtained with the drone shooting distance as the radius;

[0013] (7) Connect the centers of all devices in the same cluster to obtain a polygon, and calculate the geometric center of the polygon as a waypoint;

[0014] (8) Determine whether all the targets to be detected on the equipment can be detected by waypoints in their respective clusters, and mark the numbers of all targets to be inspected that are not within the detection range;

[0015] (9) Delineate a circular range with the maximum detection distance of the UAV as the radius and the center of the 3D bounding box of the device as the center, and record the waypoint numbers within the circular range as alternative waypoints for the device;

[0016] (10) Determine which alternative waypoint can detect the undetected target to be inspected on the equipment;

[0017] (11) Bind the device to all the selected waypoints in a one-to-many manner and store it as a binding file;

[0018] (12) Check if any of the devices have any missing target points. If there is only one missing target point, add a waypoint directly. If there are two missing target points, set a new waypoint at the center of the line connecting the two target points. If there are multiple missing target points, set one or more new waypoints according to the relative distance to ensure that no waypoints are missed.

[0019] (13) Update and store the binding relationship between the device and waypoint.

[0020] In step (6), the density clustering algorithm DBSCAN is used to cluster the geometric centers of the equipment, as detailed below:

[0021] The DBSCAN algorithm first selects a core object in the substation (i.e., a device in the substation) as a seed, creates a cluster, and finds all its core objects. It then searches for and merges devices that are density-reachable from the core object until all core objects have been visited. A DBSCAN cluster can contain one less core object: if there is only one core object, all other non-core objects fall within the ε-neighborhood of the core object; if there are multiple core objects, at least one other core object must be within the ε-neighborhood of any core object, otherwise the two core objects cannot be density-reachable. A cluster containing too few objects can be considered noise.

[0022] Compared with the prior art, the advantages of the present invention are as follows:

[0023] 1. Unlike existing technologies that rely on manual waypoint drawing, this invention automatically generates inspection waypoints through equipment clustering, significantly improving the efficiency of waypoint generation.

[0024] 2. The waypoints in this invention are generated based on the geometric center coordinates of each class. Therefore, the number of waypoints will not be redundant and will cover each device exactly. This invention can collect images of all devices in the substation with the fewest waypoints, reducing the redundancy of the collected data.

[0025] 3. The types of equipment in the present invention are relatively fixed, and the targets to be inspected on each type of equipment are also the same. Therefore, by obtaining the coordinate information of all the targets to be inspected on all equipment according to the equipment type, orientation, and geometric center of the equipment, and numbering all the targets to be inspected on all equipment, it is possible to determine whether the targets to be inspected are completely covered, thus preventing the omission of targets to be inspected.

[0026] 4. This invention uses the graph-based 3D target detection depth network Graph-Detect3D to intelligently detect and generate the device type, the 3D bounding box of the target to be inspected on the device and its orientation, and utilizes the target to be inspected. Attached image description:

[0027] Figure 1 This is a flowchart of the method of the present invention.

[0028] Figure 2 This is a schematic diagram of the Graph-Detect3D structure, a graph-based 3D object detection depth network according to the present invention. Detailed implementation method:

[0029] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0030] like Figure 1 As shown, this invention provides an automatic waypoint generation method based on substation target detection clusters, including the following steps:

[0031] (1) Use drones to take aerial photos of the substation, create a 3D map of the substation, and mark the location information of each piece of equipment;

[0032] (2) Set the camera focal length, shooting distance and gimbal angle parameters, and use the drone to take pictures of all the equipment that needs to be detected;

[0033] (3) The RGB image collected in step (2) is used as the input of the graph-based 3D target detection deep network Graph-Detect3D to obtain the equipment type, the 3D bounding box of the target to be inspected on the equipment and its orientation in the substation.

[0034] (4) Calculate the geometric center of each device on the plane based on the coordinate information of the vertices of the 3D bounding box of the device in the target detection bird's-eye view output;

[0035] (5) Obtain the coordinate information of all equipment to be inspected based on the equipment type, orientation, and geometric center of the equipment, and number all equipment to be inspected.

[0036] (6) The density clustering algorithm DBSCAN is used to cluster the geometric centers of the equipment to obtain multiple clusters;

[0037] (7) Connect the centers of all devices in the same cluster to obtain a polygon, and calculate the geometric center of the polygon as a waypoint;

[0038] (8) Determine whether all the targets to be detected on the equipment can be detected by waypoints in their respective clusters, and mark the numbers of all targets to be inspected that are not within the detection range;

[0039] (9) Delineate a circular range with the maximum detection distance of the UAV as the radius and the center of the 3D bounding box of the device as the center, and record the waypoint numbers within the circular range as alternative waypoints for the device;

[0040] (10) Determine which alternative waypoint can detect the undetected target to be inspected on the equipment;

[0041] (11) Bind the device to all the selected waypoints in a one-to-many manner and store it as a binding file;

[0042] (12) Check if any of the devices have any missing target points. If there is only one missing target point, add a waypoint directly. If there are two missing target points, set a new waypoint at the center of the line connecting the two target points. If there are multiple missing target points, calculate the geometric center of the geometry after connecting them as the location of the new waypoint.

[0043] (13) Update and store the binding relationship between the device and waypoint.

[0044] The substation waypoint selection method provided by this invention employs a graph-based 3D target detection deep network to identify substation equipment and constructs waypoints based on a density clustering algorithm. Specifically, firstly, images of the substation are captured using a camera mounted on a UAV. The graph-based 3D target detection deep network Graph-Detect3D is then used to identify all equipment in the substation and all target points on those equipment. The detection results include the target's 3D bounding box, its size and position, and its orientation. This simplifies complex real-world scenarios and facilitates the rapid and smooth execution of the subsequent clustering process. Next, density clustering is performed on the equipment to obtain initial waypoints. Then, all substation equipment, waypoints, and target points on the equipment are numbered, and the correspondence between equipment and waypoints is stored. Subsequently, target points that failed to be detected are screened, and waypoints are added to cover them. Finally, the equipment and waypoint binding file is updated and stored.

[0045] Taking the graph-based 3D object detection deep network Graph-Detect3D as input, it can obtain 3D bounding boxes and orientation parameters of equipment in a substation, as well as the targets to be inspected on the equipment, for subsequent waypoint generation. Specifically, such as... Figure 2 As shown, images captured in a substation scene are used as input. The 3D object detection deep network first automatically aggregates multi-view substation image information through graph structure learning. A dynamic 3D graph is constructed between each object query and the 2D feature map to enhance object representation, especially in boundary regions. Furthermore, Graph-Detect3D employs a novel depth-invariant multi-scale training strategy, which maintains visual depth consistency by simultaneously scaling image size and object depth.

[0046] like Figure 2 As shown, Graph-Detect3D consists of three modules: an image encoder, a transformer decoder, and a target prediction head. It contains two important components: a dynamic graph feature aggregation module and a depth-invariant multi-scale training strategy. Given a set of substation scene images, Graph-Detect3D aims to predict the location and category of bounding boxes of interest. First, the image encoder (including ResNet and FPN networks) transforms these images into a set of features F at the L feature map level. Then, a dynamic 3D graph is constructed, and the dynamic graph feature aggregation module extensively aggregates 2D information to optimize the representation of the target query. Finally, the enhanced target query outputs the final predicted 3D bounding boxes of the substation equipment and the target points to be inspected on the equipment.

[0047] By performing density clustering on the center points of substation equipment, multiple clusters and their corresponding center points are obtained. Specifically, the DBSCAN density clustering algorithm is used to perform density clustering on the equipment set, resulting in multiple clusters. In this embodiment, the DBSCAN density clustering algorithm is used to cluster the center points of equipment. Specifically, the DBSCAN algorithm first selects a core object in the substation as a seed, creates a cluster, finds all its core objects, and searches for and merges devices that are density-reachable from the core object until all core objects have been visited. A DBSCAN cluster can contain one less core object: if there is only one core object, then all other non-core objects fall within the ε-neighborhood of the core object; if there are multiple core objects, then at least one other core object is within the ε-neighborhood of any core object, otherwise these two core objects are not density-reachable; a cluster containing too few objects can be considered noise. Finally, the geometric center point of the cluster is set as a waypoint.

[0048] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for automatic waypoint generation based on substation target detection clusters, characterized in that, Includes the following steps: (1) Use drones to take aerial photos of the substation, create a 3D map of the substation, and mark the location information of each piece of equipment; (2) Set the camera focal length, shooting distance and gimbal angle parameters, and use the drone to take pictures of all the equipment that needs to be detected; (3) Use the RGB image collected in step (2) as the input of the graph-based 3D target detection deep network Graph-Detect3D to obtain the equipment type, the 3D bounding box of the target to be inspected on the equipment and its orientation in the substation; (4) Calculate the geometric center of each device on the plane based on the coordinate information of the vertices of the 3D bounding box of the device in the target detection bird's-eye view output; (5) Obtain the coordinate information of all equipment to be inspected based on the equipment type, orientation, and geometric center of the equipment, and number all equipment to be inspected. (6) The density clustering algorithm DBSCAN is used to cluster the geometric center of the equipment, and multiple clusters are obtained with the drone shooting distance as the radius; (7) Connect the centers of all devices in the same cluster to obtain a polygon, and calculate the geometric center of the polygon as a waypoint; (8) Determine whether all the targets to be detected on the equipment can be detected by waypoints in their respective clusters, and mark the numbers of all targets to be inspected that are not within the detection range; (9) Delineate a circular area with the maximum detection distance of the UAV as the radius and the center of the 3D bounding box of the device as the center, and record the waypoint numbers within the circular area as alternative waypoints for the device; (10) Determine which alternative waypoint can detect the undetected target to be inspected on the equipment; (11) Bind the device to all the selected waypoints in a one-to-many manner and store it as a binding file; (12) Check if any of the devices have any missing target points. If there is only one missing target point, add a waypoint directly. If there are two missing target points, set a new waypoint at the center of the line connecting the two target points. If there are multiple missing target points, set one or more new waypoints according to the relative distance to ensure that no waypoints are missed. (13) Update and store the binding relationship between the device and waypoint.

2. The method for automatic waypoint generation based on substation target detection clusters according to claim 1, characterized in that, In step (6), the density clustering algorithm DBSCAN is used to cluster the geometric centers of the equipment, as follows: The DBSCAN algorithm first selects a core object in the substation (i.e., a device in the substation) as a seed, creates a cluster, and finds all its core objects. It then searches for and merges devices that are density-reachable from the core object until all core objects have been visited. A DBSCAN cluster must contain at least one core object: if there is only one core object, then all other non-core objects fall within the ε-neighborhood of the core object; if there are multiple core objects, then any core object must have at least one other core object within its ε-neighborhood, otherwise the two core objects are not density-reachable.

Images