Distribution network unmanned aerial vehicle autonomous inspection method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of unmanned aerial vehicles, and discloses a distribution network unmanned aerial vehicle autonomous inspection method and device, computer equipment and a storage medium. Visible light images and laser point cloud data in the modeling area range are collected, and pure laser point cloud data and pure visible light images are obtained after data preprocessing; identifying a tower component in the pure visible light image; positioning the tower component to obtain a space coordinate of the tower component in the three-dimensional space coordinate system; and generating a route sequence based on the space coordinates of the tower component in the three-dimensional space coordinate system. According to the invention, automatic and accurate route planning can be realized, the routing inspection efficiency and precision of distribution network tower components are greatly improved, the problems of low efficiency and long consumed time of manual route planning are solved, the unmanned aerial vehicle routing inspection task is more intelligent and automatic, the waste of human resources is reduced, and the safety and reliability of distribution network facilities are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a method, device, computer equipment and storage medium for autonomous inspection of distribution network unmanned aerial vehicles. Background Art

[0002] The key role of drone route planning in power grid inspection cannot be underestimated. With the development of science and technology, drones are increasingly used in the power industry, especially in the field of power grid inspection. As the premise and basis of drone inspection, drone route planning is of vital importance to improving inspection efficiency, reducing costs, and ensuring power grid security. First of all, scientific and reasonable route planning can help drones efficiently cover the entire power grid area and ensure comprehensive inspection coverage. By accurately planning the flight path, the flight trajectory can be optimized to the greatest extent, the repeated inspection area can be reduced, the inspection efficiency can be improved, and time costs can be saved; secondly, the carefully designed route planning can also help drones discover potential problems and hidden dangers in the power grid; in addition, by fully covering the target area, drones can promptly discover damage, aging or other abnormal conditions of power grid facilities, give early warnings and take necessary repair measures to ensure the safety and stability of power grid facilities.

[0003] At present, three-dimensional point cloud data is mainly obtained through lidar, and route planning is performed manually or through template matching. However, these methods have some limitations and challenges: First, manual route planning requires a lot of human resources and time costs. For complex route planning, professionals are required to perform precise calculations and planning, which may increase the cost of the project and extend the implementation cycle. Secondly, manual route planning is easily affected by subjective factors, and the planning may not be scientific and reasonable or the resources cannot be optimally utilized, which can easily lead to low route efficiency and failure to achieve the expected results. Therefore, in some route planning scenarios that require high efficiency and high accuracy, traditional manual methods may face certain limitations, and existing technologies need to be improved.

[0004] The above information is presented as background information only to assist with understanding the present disclosure and no determination or admission is made as to whether any of the above may be used as prior art with respect to the present disclosure. Summary of the invention

[0005] The present invention provides a distribution network unmanned aerial vehicle autonomous inspection method, device, computer equipment and storage medium to solve the problems existing in the prior art.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A distribution network unmanned aerial vehicle autonomous inspection method, comprising:

[0008] Determine the scope of the modeling area;

[0009] Collect visible light images and laser point cloud data in the modeling area, and obtain pure laser point cloud data and pure visible light images after data preprocessing;

[0010] Obtain a YOLOv5 neck fusion enhancement model, and identify the tower components in the pure visible light image based on the YOLOv5 neck fusion enhancement model;

[0011] Processing the pure visible light image and the pure laser point cloud data to obtain preliminary tower component positioning results;

[0012] Based on the target detection algorithm of deep learning, according to the preliminary positioning result of the tower component, the spatial coordinates of the tower component in the three-dimensional space coordinate system are determined;

[0013] A flight path sequence is generated based on the spatial coordinates of the tower components in a three-dimensional space coordinate system.

[0014] Optionally, the processing of the clean visible light image and the clean laser point cloud data to obtain a preliminary tower component positioning result includes:

[0015] Through multimodal data fusion, the pure visible light image and pure laser point cloud data obtained by identification are fused at the decision-making level to obtain preliminary tower component positioning results.

[0016] Optionally, the fusion of the recognized pure visible light image and pure laser point cloud data at the decision level includes:

[0017] The tower components are independently identified and analyzed for the pure visible light image and pure laser point cloud data, respectively, to obtain the identification results of the pure visible light image and pure laser point cloud data;

[0018] The reliability and accuracy of the recognition results based on pure visible light images and pure laser point cloud data are evaluated respectively;

[0019] According to the evaluation results of reliability and accuracy, weights are assigned to the recognition results of visible light images and laser point cloud data respectively;

[0020] Comprehensively consider the recognition results and weights of pure visible light images and pure laser point cloud data to make preliminary tower component positioning decisions;

[0021] Based on the positioning decision results, the preliminary tower component positioning results are obtained.

[0022] Optionally, the autonomous inspection method of the distribution network UAV further includes: collecting visible light images using a camera;

[0023] The processing of the pure visible light image and the pure laser point cloud data to obtain preliminary tower component positioning results includes:

[0024] The pole tower components are positioned based on the mapping relationship between the pure visible light image and the pure laser point cloud data to obtain the spatial coordinates of the pole tower components in the three-dimensional space coordinate system.

[0025] Optionally, the positioning of the tower component based on the mapping relationship between the pure visible light image and the pure laser point cloud data to obtain the spatial coordinates of the tower component in the three-dimensional space coordinate system includes:

[0026] Analyze the pure laser point cloud data to obtain current posture data;

[0027] Based on the current posture data, a mapping between the pure laser point cloud data and the pure visible light image is established to obtain the position of the tower component in the world coordinate system.

[0028] Optionally, establishing a mapping between the clean laser point cloud data and the clean visible light image based on the current posture data includes:

[0029] Through the calculation of the external parameter matrix, the coordinates of the pure laser point cloud data are converted to the camera coordinate system to obtain the point cloud in the camera coordinate system;

[0030] Projecting the point cloud onto a two-dimensional image plane according to the camera's intrinsic parameter matrix to form point cloud pixels, each pixel including its own coordinate information and depth information, the depth information representing the distance from the point cloud corresponding to the point cloud pixel to the camera;

[0031] Combining the depth information of the point cloud pixels with the coordinate information to obtain a first depth projection image containing the depth information;

[0032] A second depth projection image of the tower component is screened out from the first depth projection image, the second depth projection image is clustered, the point cluster closest to the camera is determined according to the depth value, and the point cluster closest to the camera is regarded as the point cloud corresponding to the tower component.

[0033] Optionally, clustering the second depth projection image includes:

[0034] The second depth projection images of tower components of various scales are collected and fused, and then the fused features are distributed to different layers;

[0035] The clustering result is output as a rectangular frame and a rotated rectangular frame.

[0036] Optionally, the network distribution drone autonomous inspection method further includes: obtaining a YOLOv5 neck fusion enhancement model;

[0037] The acquisition of the YOLOv5 neck fusion enhancement model includes:

[0038] Add an information aggregation and distribution module to the neck of the YOLOv5 model, which is used to globally fuse multi-level features through a collection and distribution mechanism;

[0039] The neck structure of the YOLOv5 model is optimized, the training strategy is adjusted, and the YOLOv5 neck fusion enhancement model is obtained.

[0040] Optionally, generating a route sequence based on the spatial coordinates of the tower component in a three-dimensional spatial coordinate system includes:

[0041] Construct route reference points: by determining the precise location of the tower, according to the route safety flight requirements, add safety auxiliary points above the tower as the route starting points to form the reference points of each tower;

[0042] Constructing a safety boundary: Based on the specific electrical safety requirements of the distribution network line, generate waypoint locations that meet the safety distance requirements; based on the bounding box calculated based on the point cloud cluster of the tower head, offset outward with the safety distance of the power equipment as the radius to construct a safety boundary to ensure the safety distance of the route;

[0043] Construct a waypoint sequence: First, based on the overall calculation method, the vertical height of the tower when it is wrapped is calculated according to the long side of the minimum bounding box of the OBB calculated by the mapped tower point cloud according to the FOV angle of view at the specified focal length of the drone, so that the drone is flown to a position outside the safety boundary as the drone shooting point; then, based on the overall calculation method, the drone shooting points corresponding to the insulator and disconnector are obtained in turn; finally, with the line side direction of the large size of each conductor as the forward direction, the obtained drone shooting points are sorted from left to right to generate the final route flight sequence.

[0044] On the other hand, the present invention also provides a distribution network UAV autonomous inspection device, which is used to implement the distribution network UAV autonomous inspection method as described in any one of the above items, including:

[0045] The data acquisition and processing module is used to collect visible light images and laser point cloud data in the modeling area, and obtain pure laser point cloud data and pure visible light images after data preprocessing;

[0046] A recognition module is used to obtain a YOLOv5 neck fusion enhancement model, and to identify tower components in the pure visible light image based on the YOLOv5 neck fusion enhancement model;

[0047] A positioning module, used for processing the pure visible light image and the pure laser point cloud data to obtain a preliminary tower component positioning result; and a target detection algorithm based on deep learning, and determining the spatial coordinates of the tower component in a three-dimensional space coordinate system according to the preliminary tower component positioning result;

[0048] The route generation module is used to generate a route sequence based on the spatial coordinates of the tower components in the three-dimensional space coordinate system.

[0049] On the other hand, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the autonomous inspection method for a distribution network drone as described in any one of the above items is implemented.

[0050] On the other hand, the present invention also provides a storage medium containing computer executable instructions, and the computer executable instructions are executed by a computer processor to implement the distribution network drone autonomous inspection method as described in any of the above items.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention provides a distribution network drone autonomous inspection method, device, computer equipment and storage medium, which can realize automatic and accurate route planning, greatly improve the efficiency and accuracy of distribution network tower component inspection, solve the problem of low efficiency and long time consumption of manual route planning, make drone inspection tasks more intelligent and automated, reduce the waste of human resources, and improve the safety and reliability of distribution network facilities.

[0053] The present invention has other features and advantages, which will be apparent from or will be described in detail in the accompanying drawings and the following detailed description incorporated herein, which together serve to explain the specific principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 This is a flow chart of a method for autonomous inspection of a distribution network by a UAV provided in Embodiment 1 of the present invention;

[0056] Figure 2This is a diagram of the feature fusion process of a feature pyramid network in a distribution network drone autonomous inspection method provided in the first embodiment of the present invention;

[0057] Figure 3 It is a schematic diagram of shallow layer collection and distribution in a distribution network drone autonomous inspection method provided in the first embodiment of the present invention;

[0058] Figure 4 It is a schematic diagram of deep collection and distribution in a distribution network drone autonomous inspection method provided in Embodiment 1 of the present invention;

[0059] Figure 5 This is a diagram of an improved YOLOv5 network structure in a distribution network drone autonomous inspection method provided in Embodiment 1 of the present invention;

[0060] Figure 6 It is a camera coordinate conversion image coordinate diagram in a distribution network drone autonomous inspection method provided by the first embodiment of the present invention;

[0061] Figure 7 It is a process diagram of point cloud screening based on a vision cone in a method for autonomous inspection of a distribution network UAV provided in the first embodiment of the present invention;

[0062] Figure 8 It is an algorithm flow chart of point cloud clustering of tower components in a distribution network drone autonomous inspection method provided in the first embodiment of the present invention;

[0063] Fig. 9 It is a structural block diagram of a distribution network drone autonomous inspection device provided in Embodiment 2 of the present invention.

[0064] Figure numerals: 10, data acquisition and processing module; 20, identification module; 30, positioning module; 40, route generation module. DETAILED DESCRIPTION

[0065] In order to explain in detail the possible application scenarios, technical principles, specific schemes that can be implemented, and the purposes and effects that can be achieved, the following is a detailed description of the specific embodiments listed in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.

[0066] Reference to "embodiment" herein means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or association with other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, the various technical features mentioned in the embodiments can be combined in any way to form a corresponding implementable technical solution.

[0067] Unless otherwise defined, the technical terms used in this document have the same meanings as those generally understood by those skilled in the art to which this application belongs; the use of relevant terms in this document is only for describing specific embodiments and is not intended to limit this application.

[0068] In the description of this application, the term "and / or" is an expression used to describe the logical relationship between objects, indicating that three relationships may exist, for example, A and / or B, which means: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" logical relationship.

[0069] In the present application, terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.

[0070] Without further limitations, in this application, the words "include", "comprises", "has" or other similar expressions used in the sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0071] Similar to the understanding in the Examination Guidelines, in this application, expressions such as "greater than", "less than", "exceed" and the like are understood to exclude the number itself; expressions such as "above", "below", "within" and the like are understood to include the number itself. In addition, in the description of the embodiments of this application, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically limited.

[0072] In the description of the embodiments of the present application, space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present application or facilitating the reader's understanding, and do not indicate or imply that the referred device or component must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0073] Unless otherwise expressly specified or limited, in the description of the embodiments of the present application, the terms such as "install", "connect", "connect", "fix", "set", etc. used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For technicians in the technical field to which the present application belongs, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0074] Embodiment 1

[0075] Please refer to Figure 1 An embodiment of the present invention provides a distribution network UAV autonomous inspection method, including steps S1 to S5. The following is a detailed introduction to each step of this embodiment in conjunction with the accompanying drawings.

[0076] S1. Determine the scope of the modeling area.

[0077] In this step, the detailed information of the distribution network lines is consulted, including key data such as line length, tower type, cable specifications, and the surrounding environment of the lines is fully considered, such as vegetation coverage and terrain undulations, so as to determine the scope of the modeling area.

[0078] S2. Collect visible light images and laser point cloud data in the modeling area, and obtain pure laser point cloud data and pure visible light images after data preprocessing.

[0079] In this embodiment, the drone is manually remotely controlled to fly to the distribution network line area where point cloud modeling is required. During the flight, the remote control is used to accurately control the altitude, speed and heading to ensure stable flight. Based on this, the drone obtains multi-dimensional data of the distribution network line, including visible light images and laser point cloud data, and accurately records and saves them.

[0080] Furthermore, this step collects multi-dimensional data obtained by the drone to achieve the collection of visible light images and laser point cloud data. It can be understood that effective multi-dimensional data collection will provide a solid foundation for subsequent data processing and analysis, and can achieve accurate modeling and analysis of distribution network lines.

[0081] Specifically, step S2 includes the following sub-steps:

[0082] i. Precisely control the drone to ensure its altitude, speed and heading are accurately controlled.

[0083] In this sub-step, the flight status of the drone is monitored in real time, and the altitude and speed are adjusted in time to adapt to the twists and turns of the line and the complex environment. The battery power and communication signals are closely monitored to ensure safety and stability during the flight.

[0084] ii. Obtain high-quality laser point cloud and visible light image data.

[0085] In this sub-step, the aircraft flies along the pre-specified distribution network tower line, and obtains high-quality laser point cloud and visible light image data through the mounted radar and camera equipment.

[0086] iii. Perform data preprocessing, including denoising, correction, alignment, etc.

[0087] In this sub-step, filtering algorithms (such as Gaussian filtering and median filtering) are used to remove noise and outliers in the point cloud to improve data quality. Then, distortions in the visible light image, such as lens distortion and perspective distortion, are corrected to ensure that the shape and size of objects in the image are accurate. Finally, the preprocessed laser point cloud and visible light image are aligned to the same coordinate system for subsequent data fusion.

[0088] S3. Identify tower components in pure visible light images.

[0089] In this step, the pure visible light image is input into the YOLOv5 neck fusion enhancement model for target detection to identify the tower components within the modeling area.

[0090] Furthermore, before identifying the tower components in the pure visible light image, it is also necessary to obtain the YOLOv5 neck fusion enhancement model.

[0091] Get the YOLOv5 neck fusion enhancement model including:

[0092] Add an information aggregation and distribution module to the neck of the YOLOv5 model. The information aggregation and distribution module is used to globally fuse multi-level features through a collection and distribution mechanism;

[0093] Optimize the neck structure of the YOLOv5 model, adjust the training strategy, and obtain the YOLOv5 neck fusion enhancement model.

[0094] It can be understood that, in this embodiment, by adding a GD module (gather-and-distribute) to the neck of the YOLOv5 model, the obtained YOLOv5 neck fusion enhancement model has the ability to enhance the information fusion of the neck.

[0095] Specifically, the GD module globally fuses features at different levels and injects the fused global information into each level to achieve more efficient information interaction and fusion. The advantages of this method include:

[0096] Improve the model's performance in detecting objects of different sizes, especially small objects.

[0097] Enhanced the feature fusion capability of the model neck, enabling the model to better utilize multi-scale features;

[0098] There is no significant increase in model latency.

[0099] Based on this, this embodiment can better handle targets of various sizes and shapes when performing target detection tasks based on the improved model, improve the accuracy and robustness of detection, especially in autonomous inspections of distribution network drones, which helps to more accurately identify targets such as pole tower components.

[0100] At the same time, when the YOLOv5 neck fusion enhancement model performs target detection, information at each scale is collected and fused, and then the fused features are distributed to different layers; and the target detection results are output as rectangular boxes and rotated rectangular boxes.

[0101] Please refer to Figure 2 , Figure 2 The feature fusion process of the feature pyramid network is shown. YOLOv5 consists of three parts: backbone, neck and head. Among them, the neck usually uses the feature pyramid network (FPN) and its variants to fuse multi-level features. However, when the interval layer integrates information (for example, level-1 and level-3 fusion), the traditional FPN-like structure cannot transmit information losslessly, which hinders YOLOv5 from better information fusion. Therefore, this embodiment realizes effective information exchange in YOLOv5 by introducing a "collection and distribution" mechanism to globally fuse multi-level features and inject global information into higher levels. This significantly enhances the information fusion capability of the neck without significantly increasing latency, thereby improving the performance of the model on different object sizes.

[0102] In this embodiment, the neck part of YOLOv5 is improved to avoid the information loss inherent in the original feature fusion and improve the multi-scale target recognition rate.

[0103] Specifically, in this embodiment, the identification of tower components is achieved through the following steps:

[0104] i.Backbone extracts features.

[0105] Specifically, a backbone network is used to extract image features, such as ResNet-50, Darknet53, etc., and the extracted features are processed, such as Focus slicing processing, CSPDarknet53 feature extraction, etc.

[0106] Specifically, in the object detection task, a backbone network is usually used to extract image features. They have been pre-trained on large datasets (such as ImageNet, COCO, etc.) and are convolutional neural networks with pre-trained parameters, such as ResNet-50, Darknet53, etc. The key steps are as follows:

[0107] (1) Focus Slicing Processing

[0108] CSPDarknet53 performs neighboring downsampling before extracting features to generate four complementary but different sub-images, converting width and height information into channel dimension information. After the four images are concat-operated, a new image with 12 channels is obtained, and then a double downsampled feature map with no information loss is obtained through a Conv operation. Assuming the image shape is Nx3xHxW, a slicing operation is used to convert it into a feature map of Nx12xH / 2xW / 2, and then a Conv operation is used to convert it into a feature map of Nx32xH / 2xW / 2.

[0109] (2) CSPDarknet53 feature extraction

[0110] The C3 module is a key component of CSPDarknet53, which contains 3 standard convolutional layers and multiple BottleNeck modules. The BottleNeck module draws on the residual structure of ResNet, where the left branch halves the number of channels of the feature map through Conv1x1, and then doubles the number of channels through Conv3x3 convolution to extract features. The right branch is connected to the residual through shortcut and added to the output features of the left branch. In the end, while effectively solving the gradient problem, 4 features of different scales are extracted.

[0111] ii. Optimized Neck fusion features.

[0112] In this step, the “gather and distribute” mechanism is introduced to globally fuse multi-level features and inject global information into higher levels to achieve effective information exchange in YOLOv5. Then, the GD (gather-and-distribute) mechanism is used to collect and fuse information at each scale, and the fused features are distributed to different layers to avoid the inherent information loss of the traditional FPN structure and alleviate the problem of information loss caused by the layer-by-layer fusion of the traditional feature pyramid FPN.

[0113] Although YOLOv5 uses FPN and PAN to perform multi-scale feature fusion, the information fusion between the interval layers needs to pass through the intermediate layer. This transmission mode causes a lot of information loss during the calculation process, so the GD (gather-and-distribute) mechanism is used to collect and fuse information at each scale, and then distribute the fused features to different layers to avoid the inherent information loss of the traditional FPN structure. The GD collection-distribution process includes three modules: feature alignment module (FAM), feature information fusion module (IFM), and feature information distribution module (Inject). The specific steps are as follows:

[0114] (1) Input: Assuming the image shape is Nx3xHxW, the backbone extracts four multi-scale features, namely B2 (NxCb2xH / 4xW / 4), B3 (NxCb3xH / 8xW / 8), B4 (NxCb4xH / 16xW / 16), and B5 (NxCb5xH / 32xW / 32). Where N represents the batch size, Cb represents the number of channels of feature maps of different scales, and Rbi represents the height and width of feature maps of different scales.

[0115] (2) Shallow collection and distribution: please refer to Figure 3 , Figure 3 The principle of shallow collection and distribution is shown. Low-GD is mainly used to fuse the feature information of the shallow layer of the model, including the above-mentioned FAM and IFM modules, and retain the high-resolution features of small target information. The Low-FAM module normalizes B2, B3, B4, and B5 to the size of [h / 4, w / 4] of B4, and then Concat to obtain the feature Falign (NxCb4xH / 16xW / 16).

[0116] F align =Low_FAM([B2,B3,B4,B5])

[0117] Low-IFM includes Conv, RepBlock modules, and Split operations. Conv is used to adjust the channel to adapt to the size of different models. Falign is input into the RepBlock module to obtain the Ffuse fusion feature. Ffuse is split into Finj_P3 and Finj_P4 on the channel through split, and then the next step of feature fusion is performed with different levels. The formula is as follows:

[0118] F fuse =RepBlock(F align )

[0119] F inj_P3 ,Finj_P4 =Split(F fuse )

[0120] The Inject module is divided into Inject and Inject with LAF module. The features B3 and B4 at the current scale, as well as the global features Finj_P3 and Finj_P4 generated by Low-GD and Low-IFM are used as input, and the size is normalized through bilinear interpolation or average pooling. Finally, the features P3 and P4 are extracted and fused through RepB bck.

[0121] F global_act_Pi = resize(Sigmoid(Conv act (F inj_Pi ))).

[0122] F global_embed_Pi = resize(Conv global_embed_Pi (F inj_Pi ))

[0123] F att_fuse_Pi =Conv local_embed_Pi (Bi)*F global_ac t _Pi +F global_embed_Pi

[0124] Pi=RepBlock(F att_fuse_Pi )

[0125] Taking the fusion of B3 and inj_P3 as an example, inj_P3 is divided into two branches. The left branch is processed by Conv1x1 convolution and Sigmoid, and then upsampled by 2 times by bilinear interpolation to obtain Fgbbal_act_P3. The right branch is processed by Conv1x1 and upsampled by 2 times by bilinear interpolation to obtain Fgbbal_embed_P3 as the residual branch. B3 is adjusted by Conv1x1 to obtain B3′, which is multiplied by Fgbbal_act_P3 and then added to Fgbbal_embed_P3 to obtain Fatt_fuse_P3. Finally, it is enhanced by RepBlock to obtain P3.

[0126] (3) Deep collection and distribution: please refer to Figure 4 , Figure 4 The principle of deep acquisition and distribution is shown. High-GD is mainly used to fuse the feature information of the deep layer of the model, including the above-mentioned FAM and IFM modules, and retain the low-resolution features of large target information. The High-FAM module normalizes P3, P4, and P5 to the size of [h / 32, w / 32] of P5, and then Concat to obtain the feature Falign (NxCb5xH / 32xW / 32).

[0127] F align =High_FAM([P3, P4, P5])

[0128] High-IFM includes Conv, Transformer modules, and Split operations. Conv is used to adjust the channel to adapt to the size of different models. Falign is input into the Transformer module to obtain the Ffuse fusion feature. Ffuse is split into Finj_N4 and Finj_N5 on the channel through split, and then the next step of feature fusion is performed with different levels. The formula is as follows:

[0129] F fuse = Transformer(F align )

[0130] F inj_N4 , F inj-N5 =Split(Conv1×1(F fuse ))

[0131] The Inject module is divided into Inject and Inject with LAF module. The features P4 and P5 at the current scale, as well as the global features Finj_N4 and Finj_N5 generated by Low-GD and Low-IFM are used as input, and the size is standardized through the average pooling operation. Finally, the features N3 and N4 are extracted and fused through RepBlock.

[0132] F global_act_Ni = resize(Siggmoid(Conv act (F inj_Ni )))

[0133] F global_embed_Ni = resize(Conv global_embed_Ni (F inj_Ni ))

[0134] F att_fuse_Ni =Conv local_embed_Ni (Pi)*F global_act_Ni +F global_embed_Ni

[0135] Ni=RepBlock(F att_fuse_Ni )

[0136] Taking the fusion of P4 and inj_N4 as an example, inj_N4 is divided into two branches. The left branch is processed by Conv1x1 convolution and Sigmoid, and then upsampled by 2 times bilinear interpolation to obtain Fglobal_act_N4. The right branch is processed by Conv1x1 and upsampled by 2 times bilinear interpolation to obtain Fglobal_embed_N4 as the residual branch. P4 is adjusted by Conv1x1 to obtain P4', which is multiplied by Fgloball_act_N4 and then added to Fglobal_embed_N4 to obtain Fatt_fuse_N4. Finally, it is enhanced by RepBllock to obtain N4.

[0137] (4) Output: Assuming the image shape is Nx3xHxW, the neck fusion feature obtains three multi-scale features, namely N3 (NxCb3xH / 8xW / 8), N4 (NxCb4xH / 16xW / 16), and N5 (NxCb5xH / 32xW / 32). Among them, N represents the batch size, Cbi represents the number of channels of feature maps of different scales, and Rbi represents the height and width of feature maps of different scales.

[0138] iii.head classification regression.

[0139] Specifically, this step uses grid-based anchors to perform target detection on feature maps of different scales, locates and classifies the detection results, calculates the positioning loss, classification loss, and confidence loss, and then reshapes the data format as needed, and activates the original grid coordinates accordingly.

[0140] like Figure 5 As shown, Figure 5 The improved YOLOv5 network structure is shown in Figure 1. The main part of the head is 3 Detect detectors, whose network structure consists of only three Conv1x1, corresponding to three detection feature layers. It uses grid-based anchors to perform target detection on feature maps of different scales. The main process is: after obtaining the output, compare it with the real data annotation, calculate the positioning loss, classification loss and confidence loss, then reshape the data format as needed, and activate the original grid coordinates accordingly.

[0141] (1) Positioning and classification

[0142] The feature map is deeply integrated through the Conv operation for subsequent target positioning and classification. Then, the coordinate information of the target box and the confidence of the category are gradually extracted using multiple layers of Conv and CIOU_LOSS. The final output is usually a tensor of a specific size, in which each element corresponds to a prediction box, including the bounding box coordinates, target confidence, and the probability of each category.

[0143] After the above feature fusion, three feature maps N3, N4, and N5 are obtained. For each feature map, 1 Conv1x1 is used to adjust the number of channels. The final number of channels is related to the number of categories that need to be distinguished. There are 3 candidate boxes for each feature point on each feature map. For example, the COCO dataset contains 80 categories, and the final channel dimension should be 255=3x(4+1+80). Among them, 3 means that there are 3 candidate boxes at each position, and each candidate box has 4 position parameters, 1 confidence parameter, and 80 category parameters.

[0144] (2) Non-maximum suppression

[0145] In the target detection task, a large number of candidate boxes will be generated at the same target location, and these candidate boxes may overlap, so it is necessary to use non-maximum suppression to find the best target bounding box and eliminate redundant bounding boxes. Specific steps:

[0146] ① Sort the confidence of all prediction boxes in descending order;

[0147] ② Select the prediction box with the highest confidence and calculate the IOU value with other prediction boxes;

[0148] ③ Remove the prediction boxes with high overlap, and delete them directly if IOU>threshold;

[0149] ④The prediction box with the highest confidence is considered to be correctly predicted, and the remaining prediction boxes return to step ①.

[0150] Based on the above content, this embodiment has the following advantages over the prior art in identifying tower components:

[0151] 1. Accurate recognition results: Input an image to the improved YOLOv5 model, and it can output an image with the bounding boxes of the tower components marked.

[0152] 2. High algorithm performance: The original YOLOv5 outputs are all rectangular boxes, while the outputs in the target detection field in this embodiment are divided into rectangular boxes and rotated rectangular boxes, and the algorithm performance is higher.

[0153] 3. Improve recognition rate: This embodiment enhances multi-scale feature fusion, which can improve the recognition rate and make the output result more accurate than the original YOLOv5.

[0154] 4. Accurate position determination: According to the camera imaging principle, rays can be emitted from the 2D image to the 3D space according to the camera position and pixel position, and all 3D points passed by the rays can be projected to the pixel position on the image.

[0155] S4. Position the tower components to obtain the spatial coordinates of the tower components in the three-dimensional space coordinate system.

[0156] The purpose of this step is to locate the tower components based on the mapping relationship between the pure visible light image and the pure laser point cloud data, obtain the spatial coordinates of the tower components in the three-dimensional space coordinate system, and restore the position in the world coordinate system by combining the coordinate information of the distribution network tower components identified in the visible light image, using the camera parameters and the three-dimensional information of the laser point cloud for registration. Specifically, by fusing the pixel position of the distribution network tower components in the image with the three-dimensional information of the laser point cloud, the internal and external parameters of the camera and the spatial projection technology are used to convert the coordinate information of the distribution network tower components identified in the visible light image into a three-dimensional position in the world coordinate system.

[0157] Specifically, this step obtains a preliminary pole tower component positioning result by processing the pure visible light image and the pure laser point cloud data; and determines the spatial coordinates of the pole tower component in the three-dimensional space coordinate system according to the preliminary pole tower component positioning result based on the deep learning target detection algorithm.

[0158] Specifically, in this embodiment, the following steps are used to achieve preliminary positioning of the tower components and acquisition of the spatial coordinates of the tower components in the three-dimensional space coordinate system:

[0159] Analyze the pure laser point cloud data and obtain the current posture data;

[0160] Based on the current posture data, a mapping between pure laser point cloud data and pure visible light images is established to obtain the position of the tower components in the world coordinate system.

[0161] It can be understood that by analyzing and processing the image data, the elevation, longitude and latitude, and Euler angles corresponding to the visible light image, as well as the focal length, optical center, and distortion parameters can be accurately obtained.

[0162] Among them, when establishing the mapping between pure laser point cloud data and pure visible light image, the coordinates of the pure laser point cloud data are converted to the camera coordinate system through external parameter matrix calculation to obtain the point cloud in the camera coordinate system; then, the point cloud is projected to the two-dimensional image plane according to the camera's internal parameter matrix to generate a projection image; then, the depth value of the point cloud is merged with the pixel coordinates of the projection image to obtain a first depth projection image containing depth information; finally, the second depth projection image of the tower component is screened out from the first depth projection image, the second depth projection image is clustered, and the point cluster closest to the camera is judged according to the depth value, and it is regarded as the point cloud corresponding to the tower component.

[0163] The following is a detailed introduction to each sub-step included in this step.

[0164] i. Calculation of external parameter matrix.

[0165] First, the camera coordinates are converted to the UTM coordinate system based on the WGS84 coordinate system to ensure that they are in the same coordinate system as the laser point cloud to fully understand the position and direction of the camera in space. Then, the UTM band converted from WGS84 is calculated and converted into UTM coordinates to obtain the internal characteristics of the camera and the camera pose. Next, the rotation matrix from ENU to NED and the rotation matrix from NED to the camera coordinate system are calculated; finally, the laser point cloud is converted from the UTM coordinate system to the camera coordinate system.

[0166] The camera coordinates are based on the WGS84 coordinate system and need to be converted to the UTM coordinate system to ensure that they are in the same coordinate system as the laser point cloud. Therefore, the UTM zone converted from WGS84 is calculated according to the following formula and converted into UTM coordinates. This processing helps to achieve effective fusion and registration of images and laser point clouds, while also ensuring the consistency of image pose and spatial geographic information, providing more accurate and reliable data support for subsequent map making, navigation and positioning tasks.

[0167]

[0169] Usually, the UTM coordinate system of the laser point cloud uses the North East Sky (ENU) coordinate system, while the Euler angle parameters provided by DJI drones are based on the North East Earth (NED) coordinate system. Therefore, first calculate the rotation matrix R-1 from ENU to NED and the rotation matrix R-2 from NED to the camera coordinate system according to the following formula, and then multiply them to obtain the rotation matrix from ENU to the camera coordinate system, which is used to convert the laser point cloud from the UTM coordinate system to the camera coordinate system.

[0170]

[0171] Among them, α represents the yaw angle of the camera, β represents the pitch angle of the camera, and γ represents the roll angle of the camera. Then, the UTM coordinate T = (, Xc., Yc., Zc.) is calculated based on the WGS84 coordinates provided by the image, and multiplied by the rotation matrix R to obtain the translation matrix t = -R·T, which is used to convert the laser point cloud from the UTM coordinate system to the camera coordinate system. The calculation formula is as follows, where Pw represents the laser point cloud in the UTM coordinate system, Pc represents the laser point cloud in the camera coordinate system, and the rotation matrix R and the translation matrix t are collectively referred to as external parameters.

[0172] P c =R·P w +t

[0173] ii. Calculation of internal parameter matrix.

[0174] In this sub-step, first read the XS24P001437 information to obtain the focal length, optical center, distortion parameters, and longitude, latitude, elevation and other parameters of the camera, then calculate the focal length of the camera on the x-axis and y-axis, and the center coordinates of the image; then convert the camera coordinates into projection coordinates, and then convert them into pixel coordinates; finally, perform two-dimensional projection on the point cloud to realize the positioning of the tower components.

[0175] Please refer to Figure 6 , Figure 6 The conversion of camera coordinates to image coordinates is shown in the figure. The visible light image taken by the DJI M350 drone equipped with the L1 gimbal can obtain the focal length, optical center, distortion parameters, as well as longitude, latitude, elevation and other parameters by reading the XS24P001437 information. The process of converting camera coordinates to image coordinates is shown in the figure. The conversion from camera coordinates Pc (X, Y, Z) to projection coordinates p (x, y.) can be achieved in the x-axis and y-axis directions through the "similar triangle" principle. Then, according to the image center coordinates, cx. and cy, the projection coordinates p (x, y.) are converted to pixel coordinates p (u, v.).

[0176] The projection coordinate system takes the center of the image as its origin, while the pixel coordinate system takes the upper left corner of the image as its origin. The calculation formulas for image coordinates and pixel coordinates p(u,v.) are as follows: fx and fy are the focal lengths of the camera on the x-axis and y-axis, and cx and cy represent the center coordinates of the image in pixels. Arranging the following formula x, we can get the matrix expression p=K·Pc, where K is the intrinsic parameter matrix of the camera, which is used to project the three-dimensional coordinates onto the 2d plane.

[0177]

[0178] This step takes into account the camera’s pose information to more accurately match and process the laser point cloud with the image data, thereby achieving precise positioning of the tower components, which in turn helps improve the accuracy of subsequent analysis and applications.

[0179] iii. Two-dimensional projection of point cloud.

[0180] First, the laser point cloud is converted to the camera coordinate system through the external parameter matrix to obtain the point cloud in the camera coordinate system. Then, the point cloud is projected onto the two-dimensional image plane according to the camera's internal parameter matrix to generate a projection image; then, the depth value of the point cloud is combined with the pixel coordinates of the projection image to obtain the first depth projection image containing depth information.

[0181] In this step, the laser point cloud is converted to the camera coordinate system through the external parameter matrix to obtain the point cloud Pc(X,Y,Z) in the camera coordinate system. Subsequently, Pc(X,Y,Z) is projected to the two-dimensional image plane according to the camera's internal parameter matrix K to generate a projection image p(u,v.). Finally, the depth value Z of Pc(X,Y,Z) is merged with p(u,v.) through a connection operation to obtain the first depth projection image p(u,v.,Z) containing depth information, that is, the distance from the laser point cloud to the camera and the projected 2d pixel coordinates are stored in p(u,v.,Z).

[0182]

[0183] In this process, accurate calculation of the external parameter matrix, calibration of the internal parameter matrix and distortion parameter processing are crucial to the accuracy of the projection results, ensuring that the final projected image can accurately reflect the scene information of the real world. Through the above formula, the fusion of laser point cloud and visible light image information is achieved.

[0184] iv. Positioning of tower components.

[0185] In this step, a. based on the recognition box and depth image information in the target detection technology, the second depth projection image of the tower component is screened out, the second depth projection image is clustered, and the point cluster closest to the camera is determined according to the depth value, and it is regarded as the point cloud corresponding to the tower component. Finally, the point cloud of the tower component is denoised to reduce the interference of outliers on subsequent processing.

[0186] Based on the recognition frame and depth image information in the target detection technology, it is possible to accurately locate and restore each component inside the tower structure in three-dimensional space. The specific steps are as follows:

[0187] (1) Point cloud screening of tower components

[0188] Please refer to Figure 7 , Figure 7The process of filtering point clouds based on the cone of vision is shown in . The pixel coordinates of the recognition box output by the target detection are box = (x, y, w, h), where x, y represent the pixel coordinates of the upper left corner of the recognition box, and w, h represent the width and height of the recognition box. By determining whether the pixel coordinates (u, v) of the first depth projection image p(u, v., Z) are within the recognition box, p(u, v., Z)' is filtered to represent the second depth projection image of the tower component. Afterwards, the corresponding laser point cloud Pw(U, V, W)' is obtained according to p(u, v., Z)'. However, a filtering method similar to the cone of vision will cause unnecessary interfering point clouds to be selected, such as the point cloud behind the tower component relative to the camera. Therefore, Gaussian filtering, statistical filtering and other methods are selected to denoise Pw(U, V, W)' to reduce the interference of outliers on subsequent processing.

[0189] (2) Point cloud positioning of tower components

[0190] First, cluster p(u,v.,Z)' through a clustering algorithm, and determine the point cluster closest to the camera based on the depth value Z, and regard it as the point cloud corresponding to the tower component. Specifically, the density-based DBSCAN clustering algorithm has the greatest advantage of being able to perform distance on data sets of any shape. Secondly, it does not need to specify the number of clusters before clustering, which is significantly different from the K-MEANS algorithm. It can effectively identify multiple groups of point cloud clusters with sufficient density and exclude isolated points in sparse areas. The algorithm flow is as follows Figure 8 shown.

[0191] Input: second depth projection image p(u,v.,Z)'={p-1.,p-2.,…,pn}., neighborhood parameters (∈,MinPts);

[0192] ① Initialize the core object collection Initialize category k = 0;

[0193] ② Traverse the elements of p(u,v.,Z)', if it is a core object, add it to the core object set Ω;

[0194] ③If all elements in the core object set have been visited, the algorithm ends and the following steps are executed:

[0195] ④ In the core object set Ω, randomly select an unvisited core object o and mark it as visited, and mark it as category k. Finally, add the unvisited points in the neighborhood of o to the seed set Seeds;

[0196] ⑤If the seed collection Then the current cluster Ck is generated, and k=k+1, and go to step ③. Otherwise, perform the following steps:

[0197] ⑥ Select a seed from the seed set Seeds and mark it as visited, and mark its category as k. Then check whether the Seed is a core object. If so, add its unvisited points to the seed set;

[0198] ⑦ Repeat steps ⑤ and ⑥ until the seed set Seeds is empty.

[0199] Wherein, the domain is defined as the set of points whose distance to p is no greater than a given threshold ∈. Points that meet this condition are considered to be in the domain of p. If the domain of [ contains at least MinPts points, then p is considered a core object. In other words, if the density of a point reaches a specified threshold, it is considered a core object.

[0200] In addition, in another embodiment, the preliminary positioning of the tower components and the acquisition of the spatial coordinates of the tower components in the three-dimensional space coordinate system can also be achieved by the following method:

[0201] Through multimodal data fusion, the pure visible light image and pure laser point cloud data obtained by identification are fused at the decision-making level to obtain preliminary tower component positioning results.

[0202] Specifically, at the decision-making level, the fusion of the recognized pure visible light image and the pure laser point cloud data includes:

[0203] a. Perform independent tower component recognition and analysis on the pure visible light image and pure laser point cloud data obtained by recognition, and obtain recognition results of the pure visible light image and pure laser point cloud data respectively.

[0204] In this step, independent tower component recognition and analysis are performed based on pure visible light images and pure laser point cloud data. For visible light images, the YOLOv5 neck fusion enhancement model is used to detect possible tower components and provide corresponding location and category information. For laser point cloud data, features and position estimates related to tower components are extracted through specific algorithms.

[0205] b. Evaluate the reliability and accuracy of the recognition results based on pure visible light images and pure laser point cloud data respectively.

[0206] In this step, the reliability and accuracy of the recognition results obtained based on the two types of data are evaluated. For example, the credibility of each result is judged based on indicators such as the confidence of the detection and the degree of conformity with prior knowledge.

[0207] c. According to the reliability and accuracy evaluation results, weights are assigned to the recognition results of visible light images and laser point cloud data respectively.

[0208] In this step, different weights are assigned to the recognition results of visible light images and laser point cloud data based on the result evaluation, where more accurate and reliable results are given higher weights.

[0209] d. Comprehensively consider the recognition results and weights of pure visible light images and pure laser point cloud data to make preliminary tower component positioning decisions.

[0210] In this step, the recognition results of the two types of data and their corresponding weights are comprehensively considered to make the final preliminary tower component positioning decision. For example, if the detection result of the visible light image has a high confidence and is consistent with the result of the laser point cloud data within a certain range, the result of the visible light image is used as the main factor; conversely, if the result of the laser point cloud data is more reliable, it is used as the main factor. Alternatively, the location information of the two can be fused by weighted averaging or other methods.

[0211] e. Based on the positioning decision results, obtain the preliminary tower component positioning results.

[0212] In this step, the fused preliminary positioning results are optimized and adjusted to improve their accuracy and stability, for example, by removing possible outliers or reducing the impact of noise through smoothing.

[0213] S5. Generate a flight path sequence based on the spatial coordinates of the tower components in the three-dimensional space coordinate system.

[0214] The purpose of this step is to combine the relative position characteristics of the tower component coordinates with the specific requirements of the distribution network photography points, construct drone photography points outside the tower bounding box, and automatically generate a route sequence.

[0215] It is understandable that, based on the determined coordinates of the tower components and the relevant location features, appropriate drone photo points are constructed around the towers according to the specific requirements of the distribution network photo points. The setting of these photo points needs to consider factors such as shooting angle, distance, and coverage to ensure that the required information can be obtained comprehensively and accurately. These photo points are reasonably sorted and connected through algorithms to form an optimal route sequence, so that the drone can efficiently complete the shooting task according to the planned route while ensuring the safety and effectiveness of the flight.

[0216] In this embodiment, the step includes:

[0217] Construct route reference point: Construct route reference point by determining the starting point of the route;

[0218] Constructing a safety boundary: Based on the bounding box calculated based on the point cloud cluster of the tower head, the safety boundary is constructed by offsetting outward with the safety distance of the power equipment as the radius;

[0219] Construct a waypoint sequence: Based on the coordinates of the center points of each tower component, construct the drone photography position according to the predetermined angle rules, and detect the positional relationship between the waypoint position and the safety boundary in real time to ensure that the waypoint is outside the boundary. Finally, sort the waypoints in a counterclockwise direction to obtain a waypoint sequence.

[0220] More specifically, the detailed steps of the route sequence are as follows:

[0221] i. Construct route reference points.

[0222] In this step, a certain distance threshold is added to the elevation of the tower vertex as the start and end points of the drone route Ps(x1,y1,z1+H) and Pe(x1,y1,z1+H).

[0223] ii. Construct a safety boundary.

[0224] In this step, the OBB bounding box is calculated based on the point cloud cluster of the tower head, and the safety distance of the power equipment is used as the radius. A new safety boundary is constructed by offsetting outward. The route needs to be outside the safety boundary and the line direction.

[0225] iii. Construct a waypoint sequence.

[0226] In this step, the coordinates of the center points of the tower head, crossarm, insulator and other components are used as the shooting reference, and the drone shooting position is constructed according to certain angle rules (horizontal, vertical, overlooking, etc.), and the position relationship between the waypoint position and the safety boundary is detected in real time to ensure that the waypoint is outside the boundary. Finally, the waypoints are sorted in a counterclockwise direction to obtain a set of waypoint sequences, thereby realizing the automatic generation of the route.

[0227] In this embodiment, a distribution network tower component identification and positioning method is implemented based on target detection and laser point cloud. By combining target detection with laser point cloud, the distribution network tower component identification and positioning are realized, thereby realizing the transition from manual route planning to automatic route planning, which can solve the problems of low efficiency and long time consumption of manual route planning. The distribution network drone autonomous inspection method provided in this embodiment comprehensively utilizes visual and laser data, which can promote the development of the distribution network industry towards intelligence and automation, and improve the operating efficiency and service quality of urban power distribution systems.

[0228] It can be seen from the above embodiments that, compared with the prior art, the advantages of the present invention are:

[0229] 1) No need to collect data repeatedly: Repeated data collection consumes time and resources, and may introduce unnecessary errors. Using data collected once can meet the subsequent component identification and spatial position mapping needs, which not only improves efficiency, but also reduces interference with the environment and equipment, helps to reduce costs and ensure data consistency and reliability.

[0230] Fast and high-precision recognition: The “collection-distribution” module collects and fuses information at all scales, and then distributes the fused features to different layers, which not only avoids the inherent information loss of the traditional FPN structure, but also enhances the information fusion capability of the Neck part without significantly increasing the latency. Ultimately, an ideal balance between latency and accuracy is achieved.

[0231] 2) Accurately restore the three-dimensional position: Combine the camera parameters and the three-dimensional information of the laser point cloud for registration, and use the DBSCAN clustering algorithm to accurately restore the three-dimensional spatial position of the tower components. This is conducive to data visualization analysis, a more intuitive understanding of the equipment location and status, and a more intuitive reference for decision-making. In addition, it can quickly and accurately locate the site to improve work efficiency.

[0232] 3) Support automatic route planning: By providing accurate component coordinates, it supports more sophisticated and accurate automatic route planning. Compared with traditional template-based automatic route planning, the planned path can be adjusted according to the actual location of the distribution network tower components to ensure the comprehensiveness and accuracy of drone inspections. It also reduces invalid inspection data and improves utilization efficiency.

[0233] Embodiment 2

[0234] Please refer to Fig. 9 Based on the above embodiments, an embodiment of the present invention provides a distribution network UAV autonomous inspection device, which is used to implement the distribution network UAV autonomous inspection method described in the above embodiments.

[0235] Specifically, the device includes:

[0236] The data acquisition and processing module is used to collect visible light images and laser point cloud data in the modeling area, and obtain pure laser point cloud data and pure visible light images after data preprocessing;

[0237] The recognition module is used to identify the tower components in the pure visible light image. Specifically, the pure visible light image is input into the YOLOv5 neck fusion enhancement model for target detection, and the tower components in the modeling area are identified.

[0238] A positioning module is used to locate the tower components and obtain the spatial coordinates of the tower components in a three-dimensional space coordinate system; specifically, the positioning module is used to process the pure visible light image and the pure laser point cloud data to obtain a preliminary tower component positioning result; and a target detection algorithm based on deep learning is used to determine the spatial coordinates of the tower components in a three-dimensional space coordinate system according to the preliminary tower component positioning result;

[0239] The route generation module is used to generate a route sequence based on the spatial coordinates of the tower components in the three-dimensional space coordinate system.

[0240] Embodiment 3

[0241] Based on the foregoing embodiments, an embodiment of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the autonomous inspection method for a distribution network drone as described in the above embodiments is implemented.

[0242] Embodiment 4

[0243] Based on the foregoing embodiments, an embodiment of the present invention provides a storage medium containing computer executable instructions, and the computer executable instructions are executed by a computer processor to implement the distribution network drone autonomous inspection method as described in the above embodiments.

[0244] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of this application, this does not limit the scope of patent protection of this application. All technical solutions generated by replacing or modifying equivalent structures or equivalent processes based on the essential concept of this application using the contents recorded in the specification and drawings of this application, as well as directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are included in the scope of patent protection of this application.

Claims

1. A distribution network drone autonomous inspection method, characterized in that: include: Determine the scope of the modeling area; Collect visible light images and laser point cloud data in the modeling area, and obtain pure laser point cloud data and pure visible light images after data preprocessing; Obtain a YOLOv5 neck fusion enhancement model, and identify the tower components in the pure visible light image based on the YOLOv5 neck fusion enhancement model; Processing the pure visible light image and the pure laser point cloud data to obtain preliminary tower component positioning results; Based on the target detection algorithm of deep learning, according to the preliminary positioning result of the tower component, the spatial coordinates of the tower component in the three-dimensional space coordinate system are determined; A flight path sequence is generated based on the spatial coordinates of the tower components in a three-dimensional space coordinate system.

2. The autonomous inspection method for distribution network drone according to claim 1 is characterized in that: The processing of the pure visible light image and the pure laser point cloud data to obtain preliminary tower component positioning results includes: Through multimodal data fusion, the pure visible light image and pure laser point cloud data obtained by identification are fused at the decision-making level to obtain preliminary tower component positioning results.

3. The autonomous inspection method for distribution network drone according to claim 2 is characterized in that: The fusion of the recognized pure visible light image and the pure laser point cloud data at the decision-making level includes: The tower components are independently identified and analyzed for the pure visible light image and pure laser point cloud data, respectively, to obtain the identification results of the pure visible light image and pure laser point cloud data; The reliability and accuracy of the recognition results based on pure visible light images and pure laser point cloud data are evaluated respectively; According to the evaluation results of reliability and accuracy, weights are assigned to the recognition results of visible light images and laser point cloud data respectively; Comprehensively consider the recognition results and weights of pure visible light images and pure laser point cloud data to make preliminary tower component positioning decisions; Based on the positioning decision results, the preliminary tower component positioning results are obtained.

4. The autonomous inspection method for distribution network drone according to claim 1 is characterized in that: Also includes: Using a camera to collect visible light images; The processing of the pure visible light image and the pure laser point cloud data to obtain preliminary tower component positioning results includes: The pole tower components are positioned based on the mapping relationship between the pure visible light image and the pure laser point cloud data to obtain the spatial coordinates of the pole tower components in the three-dimensional space coordinate system.

5. The autonomous inspection method for distribution network drones according to claim 4 is characterized in that: The positioning of the tower component based on the mapping relationship between the pure visible light image and the pure laser point cloud data to obtain the spatial coordinates of the tower component in the three-dimensional space coordinate system includes: Analyze the pure laser point cloud data to obtain current posture data; Based on the current posture data, a mapping between the pure laser point cloud data and the pure visible light image is established to obtain the position of the tower component in the world coordinate system.

6. The autonomous inspection method for distribution network drones according to claim 5 is characterized in that: The establishing of a mapping between the clean laser point cloud data and the clean visible light image based on the current posture data includes: Through the calculation of the external parameter matrix, the coordinates of the pure laser point cloud data are converted to the camera coordinate system to obtain the point cloud in the camera coordinate system; Projecting the point cloud onto a two-dimensional image plane according to the camera's intrinsic parameter matrix to form point cloud pixels, each pixel including its own coordinate information and depth information, the depth information representing the distance from the point cloud corresponding to the point cloud pixel to the camera; Combining the depth information of the point cloud pixels with the coordinate information to obtain a first depth projection image containing the depth information; A second depth projection image of the tower component is screened out from the first depth projection image, the second depth projection image is clustered, the point cluster closest to the camera is determined according to the depth value, and the point cluster closest to the camera is regarded as the point cloud corresponding to the tower component.

7. The autonomous inspection method for distribution network drones according to claim 6 is characterized in that: The clustering of the second depth projection image comprises: The second depth projection images of tower components of various scales are collected and fused, and then the fused features are distributed to different layers; The clustering result is output as a rectangular frame and a rotated rectangular frame.

8. The autonomous inspection method for distribution network drones according to claim 1, characterized in that: The acquisition of the YOLOv5 neck fusion enhancement model includes: Add an information aggregation and distribution module to the neck of the YOLOv5 model, which is used to globally fuse multi-level features through a collection and distribution mechanism; The neck structure of the YOLOv5 model is optimized, the training strategy is adjusted, and the YOLOv5 neck fusion enhancement model is obtained.

9. The autonomous inspection method for distribution network drones according to claim 1, characterized in that: The generating of the route sequence based on the spatial coordinates of the tower component in the three-dimensional space coordinate system comprises: Construct route reference points: by determining the precise location of the tower, according to the route safety flight requirements, add safety auxiliary points above the tower as the route starting points to form the reference points of each tower; Constructing a safety boundary: Based on the specific electrical safety requirements of the distribution network line, generate waypoint locations that meet the safety distance requirements; based on the bounding box calculated based on the point cloud cluster of the tower head, offset outward with the safety distance of the power equipment as the radius to construct a safety boundary to ensure the safety distance of the route; Construct a waypoint sequence: First, based on the overall calculation method, the vertical height of the tower when it is wrapped is calculated according to the long side of the minimum bounding box of the OBB calculated by the mapped tower point cloud according to the FOV angle of view at the specified focal length of the drone, so that the drone is flown to a position outside the safety boundary as the drone shooting point; then, based on the overall calculation method, the drone shooting points corresponding to the insulator and disconnector are obtained in turn; finally, with the line side direction of the large size of each conductor as the forward direction, the obtained drone shooting points are sorted from left to right to generate the final route flight sequence.

10. A distribution network drone autonomous inspection device, characterized in that: A method for implementing an autonomous inspection of a distribution network by a UAV according to any one of claims 1 to 9, comprising: The data acquisition and processing module is used to collect visible light images and laser point cloud data in the modeling area, and obtain pure laser point cloud data and pure visible light images after data preprocessing; A recognition module is used to obtain a YOLOv5 neck fusion enhancement model, and to identify tower components in the pure visible light image based on the YOLOv5 neck fusion enhancement model; A positioning module, used for processing the pure visible light image and the pure laser point cloud data to obtain a preliminary tower component positioning result; and a target detection algorithm based on deep learning, and determining the spatial coordinates of the tower component in a three-dimensional space coordinate system according to the preliminary tower component positioning result; The route generation module is used to generate a route sequence based on the spatial coordinates of the tower components in the three-dimensional space coordinate system.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the distribution network drone autonomous inspection method according to any one of claims 1 to 9 is implemented.

12. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are executed by a computer processor to implement the network distribution drone autonomous inspection method as described in any one of claims 1 to 9.

Images