Unmanned aerial vehicle tree obstacle identification method based on artificial intelligence

Abstract

The invention provides an unmanned aerial vehicle tree obstacle recognition method based on artificial intelligence. The method comprises the steps that LiDAR data, image data and flight data collected by an unmanned aerial vehicle in an electric power inspection task are acquired; performing three-dimensional reconstruction based on the LiDAR data, the image data and the flight data to obtain a three-dimensional reconstruction model; identifying an electric power facility unit and a tree model unit from the three-dimensional reconstruction model; and determining whether a tree barrier exists based on the electric power facility unit and the tree model unit. Through a three-dimensional reconstruction technology and in combination with artificial intelligence, tree obstacle hidden dangers along the power transmission line are identified, and risks are reported, so that the tree obstacle hidden dangers can be cleared in time.

Description

Technical Field

[0001] The present application relates to the field of UAV power inspection technology, and more specifically, to an artificial intelligence-based UAV tree obstacle recognition method. Background Art

[0002] UAV power inspection technology is a new technology that uses drones to inspect power transmission lines and power facilities. It uses advanced drone automatic flight technology, wireless connection technology, image recognition technology, etc., and is equipped with high-definition cameras and other equipment. The drone can automatically complete the inspection of wires and power equipment. This technology relies on advanced sensors and image processing technology to visualize objects with high precision and collect data. It uses sensors such as infrared thermometers, high-definition cameras, laser radars, etc. to conduct all-round detection and monitoring of objects.

[0003] Traditional tree barrier identification relies on the judgment of manual inspections, which relies on various auxiliary forms of measurement and estimation to identify and clear potential tree barrier hazards. The development of drone power inspection technology has provided a technical basis for intelligent tree barrier identification. Of course, tree barrier identification technology also uses optical imaging and positioning technology for identification, but this identification solution is relatively inefficient.

[0004] Based on this, we propose a tree barrier identification scheme based on the three-dimensional reconstruction model of the UAV after the power inspection. Relying on the three-dimensional reconstruction technology, we design an artificial intelligence-based tree barrier identification scheme, which can effectively identify almost all tree barriers on the inspection section, so as to eliminate hidden dangers in time. Summary of the invention

[0005] The purpose of the embodiment of the present application is to provide an artificial intelligence-based drone tree obstacle recognition method, so as to identify tree obstacle hazards along the transmission line through three-dimensional reconstruction technology combined with artificial intelligence, report risks, and eliminate tree obstacle hazards in a timely manner.

[0006] In order to achieve the above purpose, the embodiments of the present application are implemented in the following manner:

[0007] In a first aspect, an embodiment of the present application provides an artificial intelligence-based drone tree obstacle identification method, including: acquiring LiDAR data, image data, and flight data collected by the drone during a power inspection mission; performing three-dimensional reconstruction based on the LiDAR data, image data, and flight data to obtain a three-dimensional reconstruction model; identifying power facility units and tree model units from the three-dimensional reconstruction model; and determining whether a tree obstacle exists based on the power facility units and the tree model units.

[0008] In combination with the first aspect, in a first possible implementation of the first aspect, the three-dimensional reconstruction model includes multiple model units, each model unit is formed by a number of continuous grid elements, and power facility units and tree model units are identified from the three-dimensional reconstruction model, including: for each model unit in the three-dimensional reconstruction model: performing feature extraction on the model unit to determine the overall features and element features of the model unit, wherein the overall features include the model position and the number of grid elements of the model unit, and the element features include the corner point position, normal direction, grid shape, grid curvature, grid area and dihedral angle of each grid element in the model unit; the overall features and element features of the model unit are used as input data and input into the recognition model to obtain the recognition results of each model unit, thereby determining the power facility unit and the tree model unit.

[0009] In combination with the first possible pair of implementation methods of the first aspect, in the second possible pair of implementation methods of the first aspect, the element features are extracted as follows: for each mesh element of the current model unit: obtain the corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle of the current mesh element; obtain the corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle of the adjacent mesh elements, wherein the adjacent mesh elements are three mesh elements that have common edges with the current mesh element; obtain the corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle of the neighboring point mesh elements, wherein the neighboring point mesh elements are three mesh elements that have common corner points with the current mesh element but are not adjacent edge mesh elements; generate the element features of the current mesh element based on the corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle of the current mesh element, three adjacent edge mesh elements and three adjacent point mesh elements.

[0010] In combination with the first possible implementation of the first aspect, in a third possible implementation of the first aspect, the recognition model includes an input layer, a feature extraction layer, a fully connected layer, and an output layer, wherein the input layer is used to receive input data, wherein the element feature is a sequence of n×r, n is the number of grid elements in the model unit, the number of grid elements in different model units is different, and n≥n t , n t is the grid element number threshold, r is the number of features that need to be extracted for each grid element, recorded as the feature dimension of the grid element; the feature extraction layer is used to process the element features of the model unit, determine the integrated features of the set length, and then splice and fuse the overall features of the model unit with the integrated features to obtain the model features of the model unit; the fully connected layer is used to perform feature mapping on the model features of the model unit to obtain the comprehensive feature vector of the model unit; the output layer is used to classify and identify the model unit based on the comprehensive feature vector to obtain the recognition result of each model unit.

[0011] In combination with the first possible implementation of the first aspect, in the fourth possible implementation of the first aspect, the feature extraction layer is a sequence network based on an attention mechanism, which is specifically used to: perform sequence feature processing on the element features of the model unit to obtain hidden features and attention weights corresponding to n time steps, wherein the dimension of each hidden feature is 1×s, and s is a set length; based on the hidden features and attention weights of n time steps, calculate the integrated features of the model unit.

[0012] In combination with the fourth possible implementation of the first aspect, in a fifth possible implementation of the first aspect, the following formula is used to calculate the integration feature of the model unit:

[0013]

[0014] Where c is the integration characteristic of the model unit, n is the number of grid elements in the model unit, and a i is the attention weight of the model unit at the i-th time step, h i is the hidden feature of the model unit at the i-th time step.

[0015] In combination with the fourth possible implementation of the first aspect, in the sixth possible implementation of the first aspect, the sequence network is RNN or a variant model of RNN, LSTM or a variant model of LSTM, GRU or a variant model of GRU.

[0016] In combination with the first possible pair of implementation methods of the first aspect, in the seventh possible pair of implementation methods of the first aspect, based on the power facility unit and the tree model unit, determining whether there is a tree barrier includes: performing linear detection between every two adjacent power facility units to determine a number of transmission cables located between the two adjacent power facility units; determining the power facility radiation area based on the power facility unit and the transmission cables; determining the tree swaying reachable area based on the tree model unit; and determining the target tree model unit with overlapping areas as a tree barrier based on the power facility radiation area and the tree swaying reachable area.

[0017] In combination with the seventh possible pair of implementation methods of the first aspect, in the eighth possible pair of implementation methods of the first aspect, two adjacent power facility units and transmission cables form a section of transmission line, and the power facility radiation area is determined based on the power facility units and the transmission cables, including: for each section of the transmission line: obtaining the voltage level of the current transmission line to determine the safety distance corresponding to the voltage level; determining the main body area of ​​the current transmission line based on the power facility units and the transmission cables; based on the main body area of ​​the current transmission line, spreading the safety distance outward in all directions to obtain the power facility radiation area of ​​the current transmission line.

[0018] In combination with the seventh possible implementation of the first aspect, in the ninth possible implementation of the first aspect, based on the tree model unit, the reachable area of ​​the tree sway is determined, including: for each tree model unit: obtaining the tree position, tree height, crown radius and number of grid elements of the current tree model unit; based on the tree height, crown radius and number of grid elements, estimating the theoretical maximum windage extension coefficient of the current tree model unit; based on the maximum windage extension coefficient, tree position and tree height, determining the reachable area of ​​the tree sway of the current tree model unit.

[0019] The beneficial effects of the present invention are as follows:

[0020] 1. This solution obtains the LiDAR data, image data and flight data collected by the drone during the power inspection mission, performs three-dimensional reconstruction, and obtains a three-dimensional reconstruction model (three-dimensional reconstruction technology has been developed very maturely. For this application, in order to better analyze the tree model unit, we recommend the use of triangular mesh reconstruction model technology). LiDAR data, image data and flight data can more comprehensively reflect the environmental information of the inspection site, including height, shape, position, etc., and can ensure the accuracy of the three-dimensional reconstruction model. Identify the power facility unit and the tree model unit from the three-dimensional reconstruction model, and then determine whether there is a tree barrier based on the power facility unit and the tree model unit, so as to clear the hidden dangers of the tree barrier in time. On the basis of the three-dimensional reconstruction model, for each model unit, rich features including corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle can be extracted from each model unit. These features can accurately reflect the three-dimensional structure and geometric properties of the model unit. The extracted features are input into the pre-trained recognition model, which can be intelligently classified and recognized based on the learned patterns of the features, so as to accurately identify the power facility units and tree model units (of course, there are many other units in the 3D reconstruction model, such as weeds, shrubs, animals, buildings, terrain, etc.). In addition, in the 3D reconstruction model, by subdividing the overall model into multiple independent model units, and each model unit is composed of several mesh elements (triangular meshes), the feature extraction and recognition process of each unit can be more accurately controlled, so that the recognition algorithm can pay more attention to local details and improve the accuracy of recognition. In addition, the recognition process can use parallel computing to identify multiple model units, effectively improving processing efficiency.

[0021] 2. In 3D reconstruction or image recognition tasks, the extraction of element features is crucial. Traditional feature extraction methods may only focus on the local features of a single grid element (such as a pixel, voxel, or grid surface), often ignoring the interaction between elements and the overall structural information. This scheme introduces rich "element features", which not only considers the characteristics of the current grid element itself, but also combines the characteristics of its neighboring grid elements and neighboring grid elements. This multi-scale and multi-level feature fusion method can more comprehensively reflect the true state of the model unit in three-dimensional space. For example, in a tree recognition task, the current grid element may represent part of a leaf, while the neighboring edge and neighboring grid elements may contain information about the trunk, branches, or other leaves. In three-dimensional space, the spatial relationship between elements is crucial to identifying and understanding the overall structure. By considering the spatial relationship of element features (combining the features of adjacent edge grid elements and adjacent point grid elements), that is, considering the spatial connection relationships such as common edges and common corners between grid elements, the shape, structure and contextual information of the model unit can be captured, which makes it easier for the feature extraction layer (sequence network) used in the recognition model that can handle variable-length sequence data to better extract features, and is also conducive to forming a more complete cognitive framework in the recognition process.

[0022] 3. Since the number of grid elements of the model unit in the 3D reconstruction model is uncertain, the input layer in the recognition model can receive sequences of different lengths and input them to the feature extraction layer according to the time step (each time step actually corresponds to each grid element), which can adapt to model units of different scales and complexities (i.e., model units with different numbers of grid elements). The feature extraction layer adopts a sequence network based on the attention mechanism (such as introducing the attention mechanism based on RNN, LSTM, GRU and other models or variant models). By calculating the attention weights at different time steps, it dynamically adjusts the influence of different grid elements on the integrated features, so that it can automatically focus on the more important feature information in the model unit, ignore irrelevant or redundant information, and improve the efficiency and accuracy of feature extraction. The integrated features are then calculated by weighted summation, making the integrated features more accurate and representative. Moreover, the dimension of the integrated features depends on the number of hidden layers in the sequence network (the dimension of the output hidden features). Therefore, it is possible to process input data of indefinite length so that the output integrated features have a uniform dimension. By integrating the features of the grid elements and concatenating and fusing them with the overall features, a more comprehensive and integrated model feature can be obtained. The fully connected layer is used to map the model features into a comprehensive feature vector, which is convenient for the output layer to perform classification and recognition, and is conducive to improving recognition accuracy.

[0023] 4. By performing linear detection between each two adjacent power facility units, the location and direction of the transmission cable can be accurately determined, providing a solid data foundation for the subsequent delineation of the radiation area of ​​the power facility. In the process of determining the radiation area of ​​the power facility, the transmission voltage of the transmission line is considered, and different safety distances are set to delineate the corresponding radiation area of ​​the power facility. This method can quickly and accurately delineate the radiation area of ​​the power facility. By estimating the theoretical maximum wind deflection extension coefficient of the tree model unit, the swaying range and possible position of the tree under extreme weather conditions can be more accurately evaluated, providing an important risk assessment basis for tree obstacle identification. In the process of estimating the tree swing reachable area, the actual situation of the tree model unit (such as position, height, crown radius, etc.) is taken into account, which can adapt to trees in different states and has strong dynamic adaptability and flexibility. At the same time, the number of grid elements is also considered (which can reflect the amount of leaves to a certain extent, thereby evaluating the degree to which the tree is affected by wind), which is conducive to more accurate estimation of the tree swing reachable area of ​​the tree model unit.

[0024] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 This is a flowchart of the tree obstacle recognition method for UAV based on artificial intelligence;

[0027] Figure 2 Schematic diagram of power inspection for drones;

[0028] Figure 3 A schematic diagram of the process of UAV conducting power inspection;

[0029] Figure 4 It is a partial schematic diagram of the three-dimensional reconstruction model;

[0030] Figure 5 Schematic diagram of the recognition model framework. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0032] See also Figure 1 , Figure 1 Flow chart of the method for identifying tree obstacles by unmanned aerial vehicles based on artificial intelligence provided in an embodiment of the present application. In this embodiment, the method for identifying tree obstacles by unmanned aerial vehicles based on artificial intelligence includes step S10, step S20, step S3 and step S40.

[0033] First, an electronic device (such as a computer, a server, etc.) may execute step S10.

[0034] Step S10: Obtain LiDAR data, image data, and flight data collected by the UAV during the power inspection mission.

[0035] In this embodiment, during the power inspection process, the drone can use the laser radar, camera, sensor, etc. on board to collect data (such as LiDAR data, image data, and flight data) in real time, and the collected data can be sent to the electronic device, so that the electronic device can obtain the LiDAR data, image data, and flight data collected by the drone during the power inspection task. Figure 2 and Figure 3 shown.

[0036] After obtaining the LiDAR data, the image data and the flight data, the electronic device may execute step S20.

[0037] Step S20: Perform three-dimensional reconstruction based on the LiDAR data, the image data and the flight data to obtain a three-dimensional reconstruction model.

[0038] In this embodiment, the electronic device can perform three-dimensional reconstruction based on these data to obtain a three-dimensional reconstructed model. Since there are already mature three-dimensional reconstruction software that meets the three-dimensional reconstruction requirements of this scenario (commercial ones such as Autodesk ReCap Pro, which supports importing data and performing automatic or semi-automatic three-dimensional reconstruction, and open source ones such as Colmap, Meshlab, etc.), they will not be described in detail here. Import the collected data into the three-dimensional reconstruction software, set the corresponding parameters, avoid too high a degree of fusion, and maintain the independence of the model units (at least the model units such as power facilities, trees, and buildings) as much as possible, perform three-dimensional reconstruction, and obtain a three-dimensional reconstructed model. Figure 4 As shown, Figure 4 A portion of the 3D reconstructed model is shown.

[0039] After obtaining the three-dimensional reconstructed model, the electronic device may execute step S30.

[0040] Step S30: identifying power facility units and tree model units from the three-dimensional reconstructed model.

[0041] In this embodiment, the three-dimensional reconstruction model includes a plurality of model units, each of which is formed by a plurality of continuous mesh elements. Then, for each model unit in the three-dimensional reconstruction model:

[0042] The electronic device can perform feature extraction on the model unit to determine the overall features and element features of the model unit, wherein the overall features include the model position and the number of mesh elements of the model unit, and the element features include the corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle of each mesh element in the model unit.

[0043] Here, in the overall feature, the number of mesh elements of the model unit is recorded as n, if n ≥ n t , then this model unit is retained for analysis, otherwise, the model unit is not analyzed (i.e., this model unit is not identified, such as grass and small debris on the terrain), where n t is the threshold of the number of grid elements (for example, 100).

[0044] The model position of the model unit can be represented by a series of corner point coordinates. For example, in each coordinate direction of the three-dimensional coordinate system in the three-dimensional reconstruction model, the maximum corner point and the minimum corner point are determined, that is, two corner point coordinates are determined in each coordinate direction, and there are a total of 6 corner points in the three coordinate directions. The coordinates of each corner point are three elements. Then, the model position can be represented by these 6 corner point coordinates (3×6=18 numerical elements, plus 1 numerical element of the number of grid elements, which is 19-dimensional features). Of course, in order to further improve the accuracy, a richer representation scheme can be adopted, for example, adding center point coordinates, and dividing each model unit into several blocks, taking the coordinates of 6 corner points in each block (two corner points in each coordinate direction). Alternatively, 20 groups of corner point pairs (or 50 groups of corner point pairs, etc.) are randomly selected, and each pair of corner points is selected in this way: a corner point that has not been selected is randomly determined, and a corner point that is farthest from this corner point is determined in the model unit, and this corner point is combined to obtain a group of focus pairs, and the one-dimensional data formed by the coordinates of each group of corner point pairs is used as the model position of the model unit.

[0045] The element features include the corner point position of each mesh element in the model unit (each mesh element has 3 corner points, each corner point has a 3D coordinate), the normal direction (expressed as a vector, (v x ,v y ,v z ), where v x 、v y 、v zRespectively representing the components of the normal vector on the x, y, and z axes), mesh shape (represented by the three internal angles of the triangular mesh), mesh curvature (represented by the curvature radius), mesh area, and dihedral angle (there are three dihedral angles, which are the plane angles between the current mesh element and the three adjacent mesh elements).

[0046] Exemplarily, the element features are extracted in the following manner:

[0047] For each grid element of the current model unit, the electronic device can obtain the corner point position (3 3D coordinates, 9 numerical elements in total), normal direction (3 numerical elements in total), grid shape (3 numerical elements in total), grid curvature (1 numerical element in total), grid area (1 numerical element in total) and dihedral angle (3 numerical elements in total) of the current grid element, as well as obtain the corner point position, normal direction, grid shape, grid curvature, grid area and dihedral angle of adjacent grid elements, where the adjacent grid elements are three grid elements that have common edges with the current grid element, and obtain the corner point position, normal direction, grid shape, grid curvature, grid area and dihedral angle of neighboring point grid elements, where the neighboring point grid elements are three grid elements that have common corner points with the current grid element but are not adjacent point grid elements.

[0048] Then, the electronic device can generate element features of the current mesh element based on the corner point positions, normal directions, mesh shapes, mesh curvatures, mesh areas and dihedral angles of the current mesh element, three adjacent edge mesh elements and three adjacent point mesh elements.

[0049] For example, the corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle are recorded in the order of the current mesh element, three adjacent mesh elements and three adjacent point mesh elements, and the three adjacent mesh elements are arranged clockwise according to the mesh element with the highest center (or the largest corner point coordinate in a certain coordinate direction), and the three adjacent point mesh elements are arranged clockwise according to the height of the common point with the current mesh element and the mesh element with the highest center. In this way, 140-dimensional features corresponding to the model unit can be formed.

[0050] Each grid element of the model unit can extract such an element feature. Then, a model unit can extract an overall feature (for example, 19 dimensions) and the element features of n grid elements (for example, 140 dimensions, denoted as r). The n grid elements can be arranged in a certain way to obtain a sequence of n×r element features of the model unit, where n is the number of grid elements in the model unit. The number of grid elements in different model units is different, and n≥n t , n t is the threshold of the number of grid elements, r is the number of features that need to be extracted for each grid element, recorded as the feature dimension of the grid element.

[0051] After extracting the model features and element features of the model unit, the electronic device can input the overall features and element features of the model unit as input data into the recognition model to obtain the recognition results of each model unit, thereby determining the power facility unit and the tree model unit.

[0052] like Figure 5 As shown, the recognition model may include an input layer, a feature extraction layer, a fully connected layer, and an output layer.

[0053] The input layer is used to receive input data, where the element feature is a sequence of n×r.

[0054] The feature extraction layer is used to process the element features of the model unit, determine the integrated features of the set length, and then splice and fuse the overall features of the model unit with the integrated features to obtain the model features of the model unit.

[0055] In this embodiment, the feature extraction layer is a sequence network based on the attention mechanism (RNN or a variant model of RNN, LSTM or a variant model of LSTM, GRU or a variant model of GRU can be selected. This embodiment takes the LSTM with the attention mechanism as an example), which is used to perform sequence feature processing on the element features of the model unit to obtain hidden features and attention weights corresponding to n time steps, where the dimension of each hidden feature is 1×s, and s is a set length (for example, 81 dimensions). Then, based on the hidden features and attention weights of n time steps, the integrated features of the model unit are calculated.

[0056] Specifically, the following formula is used to calculate the integration characteristics of the model unit:

[0057]

[0058] Where c is the integration characteristic of the model unit, n is the number of grid elements in the model unit, and a i is the attention weight of the model unit at the i-th time step, h i is the hidden feature of the model unit at the i-th time step.

[0059] The fully connected layer is used to perform feature mapping on the model features of the model unit to obtain the comprehensive feature vector of the model unit. Here, the fully connected layer can introduce nonlinear functions such as ReLU (try to avoid introducing pure Sigmoid functions, because the gradient disappearance problem can easily lead to training difficulties, but some improved Sigmoid functions such as Softplus functions are specially optimized for the gradient disappearance problem and are not subject to this restriction) to improve the nonlinear ability of the model and accelerate convergence.

[0060] The output layer is used to classify and identify the model units based on the comprehensive feature vector (classifiers such as Softmax, support vector machine or decision tree can be used) to obtain the identification results of each model unit, thereby determining the power facility unit and the tree model unit.

[0061] After identifying the power facility unit and the tree model unit, the electronic device may execute step S40.

[0062] Step S40: Determine whether there is a tree obstacle based on the power facility unit and the tree model unit.

[0063] In this embodiment, the electronic device may perform linear detection (such as Hough transformation) between every two adjacent power facility units to determine a number of transmission cables located between the two adjacent power facility units.

[0064] Afterwards, the electronic device may determine the power facility radiation area based on the power facility unit and the transmission cable.

[0065] Two adjacent power facility units and transmission cables form a section of transmission line. Then, for each section of the transmission line: the electronic device can obtain the voltage level of the current transmission line to determine the safety distance corresponding to the voltage level. For example, for power facilities, the obstacle clearance range is the overhead power line protection area, and the area within two parallel planes formed by the sidelines of the transmission cables extending horizontally outward and perpendicular to the ground. The sideline extension distance of each level of voltage conductor (i.e., safety distance, considering thunderstorm weather) can be set as follows: 5 meters for 10KV standard, 10 meters for 35KV standard, 15 meters for 110KV-330KV standard, and 20 meters for 500KV and above. The sideline extension distance here is only exemplary and can be adjusted according to different scenarios and different requirements, and is not a limitation of this application.

[0066] Based on this, the electronic device can determine the main area of ​​the current transmission line based on the power facility unit and the transmission cable, that is, the space area surrounded by the power facility unit and the transmission cable. Then, based on the main area of ​​the current transmission line, the safety distance is spread outward in all directions to obtain the power facility radiation area of ​​the current transmission line.

[0067] Afterwards, the electronic device can further determine the area accessible by the tree swaying based on the tree model unit.

[0068] For each tree model unit: the electronic device can obtain the tree position, tree height, crown radius and number of grid elements of the current tree model unit. The maximum theoretical wind deflection extension coefficient of the current tree model unit is estimated using the tree height, crown radius and number of grid elements (because the number of grid elements can reflect the luxuriance of trees to a certain extent, and thus reflect the extent of the wind force affecting the trees, so the number of grid elements is introduced here for estimation).

[0069] For example, the theoretical maximum swing area that can be achieved without breaking the tree is used as the tree swing reachable area (similar to the hemispherical area demarcated on the terrain with the root point of the tree model unit as the origin), and the wind stretch coefficient can take into account the stretching caused by the deformation of the tree branches under the influence of wind, which is mainly determined by the tree height, crown radius and number of grid elements. The details are as follows:

[0070] K j =α j β j γ j , (2)

[0071]

[0072] Among them, K j is the windage extension coefficient of the jth tree model unit, α j , β j and γ j is the influencing factor corresponding to tree height, crown radius and number of grid elements, H j is the tree height of the jth tree model unit, is the average height of other tree model units in the vicinity of the j-th tree model unit (e.g. within a radius of 10 meters), R j is the crown radius of the jth tree model unit, N j is the number of grid elements of the jth tree model unit, is the reference number of grid elements (reflecting the number of grid elements of the tree model unit when there is almost no wind impact, for example, the average number of grid elements of the tree model unit when there are no leaves on the trees).

[0073] This measurement method can take into account strong influencing factors such as tree height, crown radius and number of grid elements. Tree height essentially includes the tree height of this tree model unit and the average height of other tree model units in the adjacent area. If this tree model unit is short, the wind is mostly blocked by the surrounding trees. If this tree model unit exceeds the height of other surrounding tree model units, it will be more affected by the wind. The crown is the main area for extension, because most branches grow upward and are easily affected by the wind, thus achieving a stretching effect. The number of grid elements usually reflects the number of branches and leaves in the crown. The more the number, the greater the impact of wind.

[0074] Then, the electronic device can determine the tree swing reachable area of ​​the current tree model unit based on the maximum windage extension coefficient, the tree position and the tree height. For example, if the maximum windage extension coefficient of the current tree model unit is 1.2 and the tree height is 6 meters, then the determined theoretical tree swing reachable area is: a hemispherical area formed on the ground with the tree position (base point position on the terrain) as the sphere center and 6*1.2=7.2 meters as the swing radius, i.e., the tree swing reachable area of ​​the current tree model unit.

[0075] Ultimately, the electronic equipment can determine that the target tree model units with overlapping areas are tree barriers based on the radiation area of ​​the power facilities and the area reachable by the tree swaying, so as to provide timely warnings (and report the location coordinates of the tree barriers) and eliminate the hidden dangers of tree barriers as soon as possible.

[0076] In summary, the embodiment of the present application provides a method for identifying tree obstacles using a drone based on artificial intelligence. By acquiring the LiDAR data, image data, and flight data collected by the drone during the power inspection mission, three-dimensional reconstruction is performed to obtain a three-dimensional reconstruction model (three-dimensional reconstruction technology has been developed very maturely. For this application, in order to better analyze the tree model unit, we recommend the use of triangular mesh reconstruction model technology). LiDAR data, image data, and flight data can more comprehensively reflect the environmental information of the inspection site, including height, shape, position, etc., and can ensure the accuracy of the three-dimensional reconstruction model. The power facility unit and the tree model unit are identified from the three-dimensional reconstruction model, and then based on the power facility unit and the tree model unit, it is determined whether there is a tree obstacle so as to clear the hidden danger of the tree obstacle in time. On the basis of the three-dimensional reconstruction model, for each model unit, rich features including corner point position, normal direction, mesh shape, mesh curvature, mesh area, and dihedral angle can be extracted from each model unit. These features can accurately reflect the three-dimensional structure and geometric properties of the model unit. The extracted features are input into the pre-trained recognition model, which can be intelligently classified and recognized based on the learned patterns of the features, so as to accurately identify the power facility units and tree model units (of course, there are many other units in the 3D reconstruction model, such as weeds, shrubs, animals, buildings, terrain, etc.). In addition, in the 3D reconstruction model, by subdividing the overall model into multiple independent model units, and each model unit is composed of several mesh elements (triangular meshes), the feature extraction and recognition process of each unit can be more accurately controlled, so that the recognition algorithm can pay more attention to local details and improve the accuracy of recognition. In addition, the recognition process can use parallel computing to identify multiple model units, effectively improving processing efficiency.

[0077] In 3D reconstruction or image recognition tasks, the extraction of element features is crucial. Traditional feature extraction methods may only focus on the local features of a single grid element (such as a pixel, voxel, or grid face), often ignoring the interaction between elements and the overall structural information. This scheme introduces rich "element features", which not only considers the characteristics of the current grid element itself, but also combines the characteristics of its neighboring grid elements and neighboring grid elements. This multi-scale and multi-level feature fusion method can more comprehensively reflect the true state of the model unit in three-dimensional space. For example, in a tree recognition task, the current grid element may represent part of a leaf, while the neighboring edge and neighboring grid elements may contain information about the trunk, branches, or other leaves. In three-dimensional space, the spatial relationship between elements is crucial for identifying and understanding the overall structure. By considering the spatial relationship of element features (combining the features of adjacent edge grid elements and adjacent point grid elements), that is, considering the spatial connection relationships such as common edges and common corners between grid elements, the shape, structure and contextual information of the model unit can be captured, which makes it easier for the feature extraction layer (sequence network) used in the recognition model that can handle variable-length sequence data to better extract features, and is also conducive to forming a more complete cognitive framework in the recognition process.

[0078] Since the number of grid elements in the model unit in the 3D reconstruction model is uncertain, the input layer in the recognition model can receive sequences of different lengths and input them to the feature extraction layer according to time steps (each time step actually corresponds to each grid element), which can adapt to model units of different scales and complexities (i.e., model units with different numbers of grid elements). The feature extraction layer uses a sequence network based on the attention mechanism (such as introducing the attention mechanism based on RNN, LSTM, GRU and other models or variant models). By calculating the attention weights at different time steps, it dynamically adjusts the influence of different grid elements on the integrated features, so that it can automatically focus on the more important feature information in the model unit, ignore irrelevant or redundant information, and improve the efficiency and accuracy of feature extraction. The integrated features are then calculated by weighted summation, making the integrated features more accurate and representative. Moreover, the dimension of the integrated features depends on the number of hidden layers in the sequence network (the dimension of the output hidden features). Therefore, it is possible to process input data of indefinite length so that the output integrated features have a uniform dimension. By integrating the features of the grid elements and concatenating and fusing them with the overall features, a more comprehensive and integrated model feature can be obtained. The fully connected layer is used to map the model features into a comprehensive feature vector, which is convenient for the output layer to perform classification and recognition, and is conducive to improving recognition accuracy.

[0079] By performing linear detection between each two adjacent power facility units, the location and direction of the transmission cable can be accurately determined, providing a solid data basis for the subsequent delineation of the radiation area of ​​the power facility. In the process of determining the radiation area of ​​the power facility, the transmission voltage of the transmission line is considered, and different safety distances are set to delineate the corresponding radiation area of ​​the power facility. This method can quickly and accurately delineate the radiation area of ​​the power facility. By estimating the theoretical maximum wind deflection extension coefficient of the tree model unit, the swaying range and possible position of the tree under extreme weather conditions can be more accurately evaluated, providing an important risk assessment basis for tree obstacle identification. In the process of estimating the tree swing reachable area, the actual conditions of the tree model unit (such as position, height, crown radius, etc.) are taken into account, which can adapt to trees in different states and has strong dynamic adaptability and flexibility. At the same time, the number of grid elements is also considered (which can reflect the amount of leaves to a certain extent, thereby evaluating the degree to which the tree is affected by wind), which is conducive to more accurate estimation of the tree swing reachable area of ​​the tree model unit.

[0080] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying tree obstacles by unmanned aerial vehicles based on artificial intelligence, characterized in that: include: Obtain LiDAR data, image data, and flight data collected by drones during power inspection missions; Perform three-dimensional reconstruction based on LiDAR data, image data and flight data to obtain a three-dimensional reconstruction model; identifying power facility units and tree model units from the three-dimensional reconstruction model; Based on the power facility unit and the tree model unit, determine whether there is a tree barrier.

2. The artificial intelligence-based UAV tree obstacle recognition method according to claim 1 is characterized in that: The 3D reconstruction model includes multiple model units, each of which is formed by a number of continuous mesh elements. The power facility unit and the tree model unit are identified from the 3D reconstruction model, including: For each model unit in the 3D reconstructed model: extract features of the model unit to determine the overall features and element features of the model unit, wherein the overall features include the model position and the number of mesh elements of the model unit, and the element features include the corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle of each mesh element in the model unit; The overall characteristics and element characteristics of the model unit are used as input data and input into the recognition model to obtain the recognition result of each model unit, thereby determining the power facility unit and the tree model unit.

3. The artificial intelligence-based UAV tree obstacle recognition method according to claim 2 is characterized in that: The element features are extracted as follows: For each mesh element of the current model cell: Get the corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle of the current mesh element; Obtain the corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle of the adjacent mesh elements, where the adjacent mesh elements are three mesh elements that have common edges with the current mesh element; Get the corner point position, normal direction, mesh shape, mesh curvature, mesh area and dihedral angle of neighboring mesh elements, where the neighboring mesh elements are three mesh elements that have common corner points with the current mesh element but are not adjacent edge mesh elements; The element features of the current mesh element are generated based on the corner point positions, normal directions, mesh shapes, mesh curvatures, mesh areas and dihedral angles of the current mesh element, three adjacent edge mesh elements and three adjacent point mesh elements.

4. The artificial intelligence-based UAV tree obstacle recognition method according to claim 2 is characterized in that: The recognition model includes input layer, feature extraction layer, fully connected layer and output layer. The input layer is used to receive input data, wherein the element feature is a sequence of n×r, n is the number of grid elements in the model unit, the number of grid elements in different model units is different, and n≥n t , n t is the threshold value of the number of grid elements, r is the number of features to be extracted for each grid element, recorded as the feature dimension of the grid element; The feature extraction layer is used to process the element features of the model unit, determine the integrated features of the set length, and then splice and fuse the overall features of the model unit with the integrated features to obtain the model features of the model unit; The fully connected layer is used to perform feature mapping on the model features of the model unit to obtain a comprehensive feature vector of the model unit; The output layer is used to classify and identify the model units based on the comprehensive feature vector to obtain the identification result of each model unit.

5. The artificial intelligence-based UAV tree obstacle recognition method according to claim 2 is characterized in that: The feature extraction layer is a sequence network based on the attention mechanism, which is specifically used for: Perform sequence feature processing on the element features of the model unit to obtain the hidden features and attention weights corresponding to n time steps, where the dimension of each hidden feature is 1×s, and s is the set length; Based on the hidden features and attention weights of n time steps, the integrated features of the model unit are calculated.

6. The artificial intelligence-based UAV tree obstacle recognition method according to claim 5 is characterized in that: The integration characteristics of the model unit are calculated using the following formula: Where c is the integration characteristic of the model unit, n is the number of grid elements in the model unit, and a i is the attention weight of the model unit at the i-th time step, h i is the hidden feature of the model unit at the i-th time step.

7. The artificial intelligence-based UAV tree obstacle recognition method according to claim 5 is characterized in that: The sequence network is RNN or a variant model of RNN, LSTM or a variant model of LSTM, GRU or a variant model of GRU.

8. The artificial intelligence-based UAV tree obstacle recognition method according to claim 2, characterized in that: Determine whether there are tree barriers based on the power facility unit and tree model unit, including: Performing linear detection between every two adjacent power facility units to determine a number of transmission cables located between the two adjacent power facility units; Determine the radiation area of ​​the power facility based on the power facility units and transmission cables; Based on the tree model unit, the tree sway reachable area is determined; Based on the radiation area of ​​power facilities and the reachable area of ​​tree sway, the target tree model units with overlapping areas are determined as tree barriers.

9. The artificial intelligence-based UAV tree obstacle recognition method according to claim 8, characterized in that: Two adjacent power facility units and transmission cables form a section of transmission line. Based on the power facility units and transmission cables, the power facility radiation area is determined, including: For each transmission line section: Obtain the voltage level of the current transmission line to determine the safety distance corresponding to the voltage level; Based on the power facility units and transmission cables, determine the main area of ​​the current transmission line; Based on the main area of ​​the current transmission line, the safety distance is spread outward in all directions to obtain the power facility radiation area of ​​the current transmission line.

10. The method for identifying tree obstacles by unmanned aerial vehicles based on artificial intelligence according to claim 8, characterized in that: Based on the tree model unit, the tree sway reachable area is determined, including: For each tree model unit: Get the tree position, tree height, crown radius and number of mesh elements of the current tree model unit; Estimate the theoretical maximum windage spread coefficient of the current tree model unit based on tree height, crown radius and number of grid elements; Based on the maximum windage extension coefficient, tree position and tree height, the tree sway reachable area of ​​the current tree model unit is determined.

Images