Optimal Path Planning Method and System for Aerial Survey of Large-Scale Buildings Based on Hopfield Network

Abstract

The present invention discloses an optimal path planning method and system for large-scale building aerial survey based on the Hopfield network, including: locating vegetation areas using near-infrared images of the survey area, estimating the surface terrain of the survey area using multi-view aerial images, and locating building areas by combining the vegetation areas and the surface terrain; extracting the edge features of the building areas, transforming the edge features into straight-line segment features, obtaining the main direction of the buildings according to the direction vectors of the straight-line segment features, and adjusting the heading to generate local flight routes for the building areas; determining the distance order matrix between buildings according to the building positions, setting the shortest path objective function and constraint conditions for global flight route optimization according to the distance order matrix, constructing the standard energy function of the Hopfield network, and iteratively solving to obtain the overall flight route covering all the buildings in the survey area. The present invention can exclude unnecessary flight areas, reduce flight losses, and can also significantly reduce the flight distance of building aerial survey tasks.

Description

Technical Field

[0001] The present invention belongs to the technical field of UAV route planning, and particularly relates to an optimal path planning method and system for large-scale building aerial survey based on the Hopfield network. Background Art

[0002] With the continuous development of sensor and control technologies, unmanned aerial vehicles (UAVs) have become an important tool for obtaining data in the field of surveying and mapping remote sensing. 0 The application scenarios of UAVs in the field of building mapping are increasing. 0 For example, in urban planning, UAVs combined with oblique photogrammetry technology can be used to construct 3D real-scene models of cities; in map production, high-precision aerial images taken by UAVs can be used for orthophoto production, digital line graph production, etc. Currently, common UAV route planning algorithms are dedicated to solving the Traveling Salesman Problem (TSP). Common solution methods include dynamic programming, genetic algorithms, ant colony algorithms, probabilistic model estimation, etc.

[0003] However, traditional planning algorithms rarely consider specific flight tasks. For example, in the task of building aerial survey, there are a large number of invalid flight areas without buildings in the route, resulting in low overall operation efficiency. Chinese Patent with Publication No. CN106403954A discloses a method for generating an automatic UAV flight path. The method is as follows: sampling the outer contour of the area to be calibrated; calculating the minimum circumscribed convex polygon of the sampled point set, and obtaining the coordinates and area of the area to be calibrated through the conversion between longitude and latitude coordinates and plane projection coordinates; obtaining the operation area based on the coordinates and area of the field and obstacles, and calculating the shortest operation path of the operation area; when the UAV operates in the operation area, determining whether the UAV can safely reach at least one safe takeoff and landing point at the next waypoint. If so, continue the operation; otherwise, return at the current waypoint. In this method, the identification of fields, obstacles, etc. requires manual participation. In addition, the generated route is a "strip-shaped" route set according to the overlap rate, which has no pertinence to the target ground objects of the aerial survey task, and there is a large flight loss when directly applied to the field of building aerial survey.

[0004] For large-scale aerial survey tasks, buildings are distributed in blocks. In order to obtain a single-unit building entity model, the route generated by the traditional route planning method will cover the entire survey area, including a large area of ​​non-building areas, which will inevitably cause unnecessary flight losses. The present invention intends to solve the terrain through the multi-view stereo geometry of the drone, automatically locate and divide the various building areas to be measured in the large-scale target survey area in combination with the NDVI results, and solve the overall shortest path connecting all building areas by constructing the Hopfield energy function, that is, through the optimal flight path planning and flight route, the overall shortest path to achieve rapid aerial survey of all building areas to be measured in the large-scale target survey area. In addition, the heading of the traditional route planning is unified. Due to the inconsistent orientation of the buildings in different block areas, it is impossible to ensure that all buildings have the best shooting angle. In response to this problem, the present invention intends to solve the main direction of the building in each building area through the improved edge operator, and adjust the flight heading of the drone in each building area according to the main direction to achieve the maximum signal-to-noise ratio, and obtain the most building information for the large-scale target survey area with the least number of aerial survey frames.

[0005] In order to meet the increasing practical demand for the use of drones in the field of building aerial surveys, and at the same time solve the problem that traditional route planning methods are not targeted at buildings and result in a large loss of time, it is very necessary to design and invent an automatic aerial survey path planning method that is targeted at building aerial survey tasks and has high operating efficiency.

[0006] The above article refers to the following references:

[0007] [1] Li Deren, Li Ming. Research progress and application prospects of UAV remote sensing system[J]. Journal of Wuhan University (Information Science Edition), 2014, 39(05): 505-513+540.

[0008] [2] Liu Qian, Liang Zhihai, Fan Huifang. A brief discussion on the development of UAV remote sensing and its industry applications [J]. Surveying and Mapping and Spatial Geographic Information, 2016, 39(06): 167-169. Summary of the invention

[0009] The purpose of the present invention is to provide a method and system for optimal path planning for aerial survey of large-scale buildings based on a Hopfield network.

[0010] The method and system of the present invention can autonomously identify the vegetation in a large-scale target survey area, locate the positions of each building area in the large-scale target survey area, exclude unnecessary flight areas in the large-scale target survey area, thereby reducing flight losses; moreover, for each building area, the main direction of the building area can be automatically estimated, and the flight heading of the unmanned aerial vehicle during aerial survey in the building area can be adjusted according to the main direction, so as to ensure that all building areas have the best shooting angles; for the overall heading covering all building areas, the overall shortest path can be autonomously solved to complete the automatic planning of the overall path. A large-scale target survey area generally includes several building areas; and a building area generally consists of a certain number M (M is an integer and M≥1) of buildings forming a building area.

[0011] The optimal path planning method for large-scale building aerial survey based on the Hopfield network provided by the present invention includes:

[0012] Using near-infrared images covering the entire large-scale target survey area to locate the vegetation area, using multi-view aerial images to estimate the surface terrain of the large-scale target survey area, and combining the vegetation area location result and the surface terrain estimation result to locate each building area in the entire large-scale target survey area;

[0013] Extracting the edge features of any building area, transforming the edge features into straight-line segment features, obtaining the main direction of the building area according to the direction vector of the straight-line segment features, adjusting the flight heading during aerial survey of the building area to be perpendicular to the main direction, and generating the local flight path of the building area according to the overlap rate requirement;

[0014] Determining the distance order matrix between building areas according to the positions of each building area in the large-scale target survey area, setting the shortest path objective function and constraint conditions for global flight path optimization according to the distance order matrix, constructing the standard energy function of the Hopfield network according to the objective function and constraint conditions, and iteratively solving the standard energy function of the Hopfield network with the minimum energy loss as the goal to obtain the overall flight path covering all building areas in the entire large-scale target survey area.

[0015] Optionally, using multi-view aerial images to estimate the surface terrain of the large-scale target survey area, specifically:

[0016] Using the unmanned aerial vehicle to take a downward photo of the entire large-scale target survey area at the take-off point, and recording the pose information of the unmanned aerial vehicle sensor at the moment of shooting;

[0017] Moving the unmanned aerial vehicle to another position to take a downward photo of the entire large-scale target survey area, and using the inertial navigation data of the unmanned aerial vehicle combined with the initial pose information to calculate the pose information at the moment of the second shooting;

[0018] Match two images according to the gray - scale feature, calculate the depth of the matching point pairs, and obtain the three - dimensional coordinate estimation of the ground points;

[0019] Use the three - dimensional coordinate estimation to fit the surface topography of the large - scale target survey area.

[0020] Optionally, combine the vegetation area positioning result and the surface topography estimation result to position each building area in the entire large - scale target survey area. Specifically:

[0021] Judge whether the pixel (x, y) meets the conditions If it meets the conditions, the pixel (x, y) belongs to the building pixel. Among them, NDVI(x, y) represents the NDVI value of the pixel (x, y), which is the vegetation area positioning result; F'(X, Y) is the surface topography estimation value; t 1 is the gray - scale threshold for distinguishing vegetation, t 2 is the height threshold for distinguishing the surface and non - surface.

[0022] Optionally, the 8 - direction enhanced edge detection operator is used to extract the edge features of the building area. The structure of the 8 - direction enhanced edge detection operator is as follows:

[0023]

[0024]

[0025] The operator kernel is denoted as g m , for the pixel (x, y), according to judge whether the pixel (x, y) is an edge pixel. Among them, t 3 is the threshold for judging whether it is an edge pixel, N 5×5 (x, y) represents the 5×5 neighborhood pixels of the pixel (x, y).

[0026] Optionally, obtain the main direction of the building area according to the direction vector of the line - segment feature, which further includes:

[0027] Calculate the direction vector of the line - segment feature. Specifically: transform the line - segment feature into the polar coordinate system, and the polar coordinate is denoted as (θ s ,ρ s ). θ s and ρ s represent the direction angle and length of the line - segment feature respectively, and take θ s as the direction vector of the line - segment feature;

[0028] Divide the line - segment features into preset different direction intervals according to the direction vector. Specifically: evenly divide the angle range [0, π] into several direction intervals; use the formula for the direction vector θ sPerform a transformation and classify the transformed direction vectors into corresponding direction intervals;

[0029] Obtain the main direction of the building area based on the direction vectors within each direction interval. Specifically: Sum the direction vectors within each direction interval to obtain a sum vector, and take the direction interval corresponding to the sum vector with the maximum modulus length as the main direction of the building area.

[0030] Optionally, the objective function is The constraint conditions are

[0031] Optionally, construct the standard energy function of the Hopfield network according to the objective function and the constraint conditions. Specifically: Transform the constraint conditions and the objective function into penalty terms of the energy function.

[0032] Optionally, the constructed standard energy function of the Hopfield network is:

[0033]

[0034] Optionally, iteratively solve the standard energy function of the Hopfield network with the goal of minimizing the energy loss, which further includes:

[0035] Step 1: Set the penalty parameters A, B, C, D and the maximum number of iterations T max , and randomly initialize the order matrix x;

[0036] Step 2: Calculate the distances between each building area to form a distance order matrix d, where d ij represents the distance from building area i to building area j;

[0037] Step 3: Determine the state transition function U 0 according to the initialized input state U ki (t):

[0038]

[0039] Step 4: Calculate the increment ΔU of the input state:

[0040]

[0041] Step 5: Update the input state and output state at the next moment using the dynamic equation and the first-order Euler method:

[0042]

[0043] Step 6: Check whether the current output state x ki satisfies the constraints, determine whether the iteration ends, otherwise return to Step 4 until the number of iterations reaches the set maximum number.

[0044] The present invention also provides a large-scale building aerial survey optimal path planning system based on the Hopfield network, comprising:

[0045] The building area positioning module is used to locate the vegetation area using the near-infrared image covering the entire large-scale target measurement area, estimate the surface topography of the large-scale target measurement area using multi-view aerial images, and locate each building area in the entire large-scale target measurement area by combining the vegetation area positioning results and the surface topography estimation results;

[0046] The single-area route planning module is used to extract the edge features of any building area, transform the edge features into straight line features, obtain the main direction of the building area according to the direction vector of the straight line features, adjust the flight heading during the aerial survey of the building area to be perpendicular to the main direction, and generate the local route of the building area according to the overlap rate requirements;

[0047] The overall survey area route optimization module is used to determine the distance order matrix between building areas according to the positions of each building area in the large-scale target survey area, set the shortest path objective function and constraints of the global route optimization according to the distance order matrix, and construct the Hopfield network standard energy function according to the objective function and constraints. The Hopfield network standard energy function is iteratively solved with the goal of minimizing energy loss to obtain the overall route covering all building areas in the entire large-scale target survey area.

[0048] The present invention has the following advantages and beneficial effects:

[0049] In view of the problem that traditional routes cannot distinguish building areas, the present invention uses drone multi-view aerial images to solve the terrain height, and combines the NDVI results of near-infrared images to automatically locate and divide each building area to be measured in the large-scale target survey area; in view of the contradiction that buildings are distributed in blocks and clusters and have different directions while the traditional route planning has a unified heading, the present invention first estimates the main direction of each building area, and then adjusts the flight heading of the drone of each building area according to the main direction; in view of the problem that the traditional route has a long invalid flight distance, a distance order matrix between all building areas in the entire large-scale target survey area is established, and then a Hopfield energy function is constructed. Through iterative optimization, the optimal flight sorties and the shortest route covering all building areas in the entire large-scale target survey area are solved. In general, the present invention can exclude unnecessary flight areas in the large-scale target survey area, thereby reducing flight losses; it can ensure that all building areas in the entire large-scale target survey area have the best shooting angle, and can also significantly reduce the flight distance of the building aerial survey task in the large-scale target survey area. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the framework of the present invention;

[0051] Figure 2 Schematic diagram of the specific process of the embodiment;

[0052] Figure 3 Near-infrared image of the measurement area and the calculation result diagram of its NDVI value in the embodiment. Among them, Figure (a) is the near-infrared image of the measurement area, and Figure (b) is the calculation result diagram of the NDVI value of the measurement area;

[0053] Figure 4 Schematic diagram of the principle of calculating the corresponding ground three-dimensional coordinate positions of the matching pixels of the multi-view aerial images of the unmanned aerial vehicle in the embodiment. In the figure, p1(u1, v1) and p2(u2, v2) represent the matching point pairs of the two images;

[0054] Figure 5 Piecewise weight function adopted in the embodiment;

[0055] Figure 6 Measurement area and its surface terrain estimation result diagram in the embodiment. Among them, Figure (a) is the near-infrared image of the measurement area, and Figure (b) is the surface terrain estimation result diagram of the measurement area;

[0056] Figure 7 Schematic diagram of the kernel structure of the 8-direction enhanced edge operator provided in the embodiment;

[0057] Figures 8 - 9 Comparison schematic diagram of extracting edges of different target buildings by using Canny operator, LoG operator, and 8-direction enhanced edge detection operator in the embodiment. Among them, Figure (a) is the target building, Figure (b) is the edge extracted by the Canny operator, Figure (c) is the edge extracted by the LoG operator, and Figure (d) is the edge extracted by the 8-direction enhanced edge detection operator;

[0058] Figure 10 Schematic diagram of the main direction estimation of the building area and the schematic diagram of generating a local flight path according to the main direction in the embodiment. Among them, Figure (a) is the schematic diagram of the main direction estimation of the building area, and Figure (b) is the schematic diagram of generating a local flight path according to the main direction;

[0059] Figure 11 Schematic diagram of the main direction estimation and local flight path generation structure of the building area in the embodiment. Among them, Figure (a) shows the main direction of the building area, and Figure (b) is the generated local flight path;

[0060] Figure 12 Schematic diagram of the process of using the Hopfield network to iteratively optimize and solve the global shortest flight path in the embodiment. Among them, Figure (a) is the global flight path optimization solution, and Figure (b) is the schematic diagram of the iterative process of the Hopfield network energy function;

[0061] Figure 13 It is a schematic diagram of the result of using the Hopfield network to iteratively optimize and solve the global shortest flight route in the embodiment. Specific implementation manners

[0062] To enable those skilled in the art to better understand the present invention, the specific implementation manners of the technical solution of the present invention will be clearly and completely described below. Obviously, what is described below is only the specific implementation manners, which do not limit the protection scope of the present invention.

[0063] It should be noted that for the sake of concise description, the "large-range target survey area" will be abbreviated as "survey area" hereinafter.

[0064] The specific implementation process of the embodiments of the present invention will be described in detail below in conjunction with the drawings. Figure 1 It is a functional framework diagram of the present invention. Figure 2 The specific process of this embodiment is as follows:

[0065] S100: Obtain the near-infrared image of the survey area, calculate the NDVI value (normalized difference vegetation index value) of each pixel, and locate the vegetation area.

[0066] The near-infrared camera can be carried by a drone to take pictures at a high altitude at the take-off point to obtain the IRRG image covering the survey area, including the drone images of the near-infrared band IR, the visible light red band R, and the visible light green band G; or the existing near-infrared images covering the survey area can also be used.

[0067] For the pixel (x, y) in the near-infrared image, calculate its NDVI value according to the following formula:

[0068]

[0069] Among them, NDVI(x, y) represents the NDVI value of the pixel (x, y), NIR(x, y) represents the near-infrared band value of the pixel (x, y), and R(x, y) represents the gray value of the red band of the pixel (x, y).

[0070] In this embodiment, the IRRG image in the Vaihingen dataset provided by the International Society for Photogrammetry is directly used as the survey area, and the near-infrared image of the survey area is shown in Figure 3 (a); perform NDVI value calculation on the survey area, and the calculation result is shown in Figure 3 (b), where the bright area represents the vegetation area and the dark area represents the non-vegetation area.

[0071] S200: Through the multi-view aerial images covering the survey area, combined with the pose provided by the drone sensor, perform gray-scale matching to calculate the depth of each pixel, so as to estimate the terrain distribution of the survey area.

[0072] In this embodiment, a drone is used to take an aerial photo of the entire survey area near the take-off point. The flight altitude is adjusted according to the size of the survey area, and the pose information of the drone sensor at the moment of shooting is recorded, specifically including the external parameter matrix R 1 and the initial position information of the photography center obtained according to GPS positioning, represented by the matrix t 1 It should be noted that the initial attitude angle is directly initialized using the drone's direction angle information. Move the drone and adjust the lens pose, and take a photo at another position, also covering the entire survey area; during this period, the pose information R 2 and t 2 at the moment of the second shooting is estimated by combining the drone's inertial navigation data with the initialized pose.

[0073] Match the two images according to the gray-scale feature, solve the depth of the matching point pairs, and thus obtain the estimated coordinates of the corresponding ground three-dimensional points. For two matching point pairs p 1 (x 1 ,y 1 ) and p 2 (x 2 ,y 2 ) on the multi-view aerial images, the estimated value Z of the ground point coordinates corresponding to this pixel point pair is solved according to the following formula:

[0074]

[0075] In formula (2), (x, y) represents the image point coordinates of the matching point pair on the image where it is located, K represents the camera internal parameter matrix, and (X, Y, Z) represents the three-dimensional coordinates of the ground point corresponding to this pixel point pair.

[0076] For multiple images (at least 2), the optimal solution of the ground point coordinates is obtained according to the least squares method.

[0077] Then, based on the estimated three-dimensional coordinates of the ground points obtained by the pixel gray-scale matching of the multi-view aerial images, the surface terrain of the survey area is estimated as follows:

[0078] First, equally weight the coordinates Z of the scattered points and fit the initial terrain surface F(X, Y); calculate the elevation residual r between the true coordinates Z of the scattered points and the initial fitted terrain surface F(X, Y) point by point:

[0079] r = Z - F(X, Y) (3)

[0080] Next, according to the size of the residual, re-weight the coordinates of the scattered points using the piecewise weight function shown in formula (4) to obtain the estimated value of the terrain surface elevation corresponding to the scattered points:

[0081]

[0082] In formula (4): W represents the weight; a and b are constant terms used to determine the slope of the weight function; g and t are elevation residual thresholds, where g takes a negative value.

[0083] It is also possible to utilize the existing Digital Terrain Model (DTM) of the target survey area to query the surface elevation of the survey area.

[0084] In this embodiment, the unmanned aerial vehicle covers the entire survey area at a high altitude above the take-off point for multi-view ground shooting, and then differentiates the pixel depth through matching point pairs. Figure 4 The schematic diagram shows the principle of resolving the corresponding ground three-dimensional coordinate positions of the matching pixels in the multi-view aerial images of the unmanned aerial vehicle. In this embodiment, the constant terms a and b of the piecewise weight function are 1 and 4 respectively, and the elevation residual thresholds g and t are -0.5 and 2.5 respectively. The piecewise weight function is shown in Figure 5 . Perform surface fitting on the survey area to obtain Figure 6 The surface terrain estimation result shown in (b).

[0085] S300: Locate each building area within the entire large-scale target survey area in combination with the vegetation area positioning result and the surface terrain estimation result.

[0086] Use the following formula to determine whether the pixel (x, y) belongs to a building pixel. When the pixel (x, y) meets formula (5), it belongs to a building pixel:

[0087]

[0088] where F'(X, Y) is the terrain surface elevation estimated in step S200; t 1 and t 2 are both thresholds. t 1 is the gray-scale threshold used to distinguish vegetation. When NDVI(x, y) < t 1 , then the pixel (x, y) does not belong to a vegetation pixel; t 2 is the height threshold used to distinguish the surface and non-surface. t 1 and t 2 are both empirical values and can be adjusted through repeated experiments and / or application effects. In this embodiment, t 1 takes the value of 158.0, and t 2 takes the value of 2.5 m.

[0089] The method of the present invention is for a large-scale target survey area, that is, "the entire survey area". There must be several building groups or building areas in this large-scale target survey area. Using steps S100 to S300, the "entire survey area" can be automatically divided into several building areas.

[0090] S400: Extract the edge features of the building area and transform the edge features into line segment features.

[0091] In this embodiment, an improved 8-direction enhanced edge detection operator is used to extract the edge features of the building area, and then the Hough transform is used to transform the edge features into line segment features.

[0092] Specifically, for the pixel (x, y), the kernel of the improved 8-direction enhanced edge detection 5×5 operator is denoted as g m , and the kernel structures of the 8-direction operators are shown in Figure 7 . Determine whether the pixel (x, y) is an edge pixel according to Equation (6), so as to extract the edge features.

[0093]

[0094] Set the threshold t for determining whether it is an edge pixel 3 . In this embodiment, t 3 takes the value of 80. When the sum of the convolution operation results of the 8 convolution kernels and the 5×5 neighborhood pixels N 5×5 (x, y) is greater than the threshold t 3 , it is determined that the pixel (x, y) is an edge pixel.

[0095] The Canny operator, LoG operator, and 8-direction enhanced edge detection operator are respectively used to detect the edge features of different target buildings, and the detection results are shown in Figures 8 - 9 . It can be seen from the figure that the Canny operator, LoG operator, and 8-direction enhanced edge detection operator can all detect the building edges, but the 8-direction enhanced edge detection operator has a better detection rate for the less obvious building edges.

[0096] S500: Calculate the direction vector of the line segment feature, and divide the line segment feature into preset different direction intervals according to the direction vector.

[0097] Calculate the direction vector of the line segment feature after the transformation in step S400, and transform it into the polar coordinate system. The polar coordinates are denoted as {(θ 1 , ρ 1 ), (θ 2 , ρ 2 )... (θ n , ρ n )}, where θ s represents the direction angle of the s-th line segment feature. Take θ s as the direction vector, and its range is [0, 2π]; ρ s represents the length of the s-th line segment feature, s = 1, 2, 3,..., n, and n represents the number of line segment features.

[0098] The angular range [0, π] is equally divided into k' direction intervals. First, transform the direction vector according to Equation (7), and then classify the transformed direction vector into the corresponding direction interval. Here, k' is a positive integer.

[0099]

[0100] In this embodiment, k' is taken as 6, so 6 direction intervals are obtained, and the line segment features are classified into the corresponding direction intervals according to the direction vectors of the line segment features.

[0101] S600: Sum the direction vectors in each direction interval respectively to obtain the sum vector, and take the direction interval corresponding to the sum vector with the maximum modulus length as the main direction of the building area; adjust the flight heading of the drone in the building area according to the main direction of the building area, and generate a local flight path according to the overlap rate requirement.

[0102] Sum the direction vectors of the k' direction intervals respectively, and the sum vector is denoted as {s 1 , s 2 ... s k'}, compare the modulus lengths of the sum vectors, and take the direction interval corresponding to the sum vector with the maximum modulus length as the main direction of the building in this area. Assume the preset flight altitude h, side overlap rate α, forward overlap rate β, camera focal length f, short-axis sensor size c1, and long-axis sensor size c2, and calculate the flight strip interval w1 and photographing interval w2 on the ground according to Equation (8).

[0103]

[0104] In this embodiment, first obtain the main direction of the building area, see Figure 10 (a); then divide the flight strips in the direction perpendicular to the main direction to generate the local flight path of the building area, see Figure 10 (b), and the dotted lines in this figure represent the generated drone flight paths. Figure 11 This is a schematic diagram of the local flight paths of each building in the measurement area in this embodiment, see the dotted lines in the figure.

[0105] S700: Determine the distance order matrix between building areas according to the positions of the building areas in the large-scale target measurement area, and determine the objective function and constraint conditions for global flight path optimization according to the order matrix.

[0106] The local flight path generated in step S600, that is, when the drone conducts aerial survey on a certain building area, it is the optimal flight path to complete the aerial survey of the building area; one local flight path corresponds to an element x in the Hopfield order matrix x of the drone flight ijCorrespondingly. Assume that the total number of building areas is N and their positions are known. Construct the Hopfield order matrix x for the drone flight. The element x in the order matrix ij represents that the drone executes the flight mission for building area j (j = 1, 2,..., N) in the i-th flight sortie (a flight sortie means flying over one building area; i represents the sequence order of a certain building area being arranged for flight, i = 1, 2,..., N). The constraint conditions are as follows:

[0107] Constraint 1: Only one neuron in each row of the order matrix x is activated, that is, each building area is flown exactly once.

[0108] Constraint 2: Only one neuron in each column of the order matrix x is activated, that is, the drone cannot execute tasks for two or more areas simultaneously.

[0109] Constraint 3: The sum of the number of all activated neurons in the order matrix x is equal to the total number of all building areas, that is, the drone needs to exactly complete the flight missions for all building areas.

[0110] The three constraint conditions are as follows:

[0111]

[0112] The goal of the overall measurement area path optimization of the drone is to minimize the sum of all path lengths. Therefore, the objective function for solving the shortest path is shown in Equation (10), where d represents the distance matrix, and d ik represents the distance from the i-th flight area (i.e., the corresponding building area flown in the i-th flight sortie) to the k-th flight area (i.e., the corresponding building area flown in the k-th flight sortie, where k = 1, 2,..., N and k ≠ i).

[0113]

[0114] In this embodiment, the number of building areas is 7, and the order matrix is a 7×7 matrix; when solving the distance order matrix, the distance between building areas is defined as the distance between the centers of the minimum circumscribed rectangles of the building areas. The distance schematic is shown in Figure 12 (a).

[0115] S800: Construct the standard energy function of the Hopfield network according to the objective function and constraint conditions.

[0116] In this embodiment, the three constraint conditions and the objective function of the shortest total path length are transformed into penalty terms of the energy function to obtain the general form of the overall energy function E, as shown in Equation (11):

[0117]

[0118] Among them, the first three terms are constraint terms, and the fourth term is the objective function; A, B, C, and D are all penalty parameters and are fixed values; x il represents that the drone executes the flight mission in the building area l (l = 1, 2,..., N and l ≠ i) in the i-th flight sortie. The general form of the energy function of the route optimization problem described in Equation (11) is corresponded to the standard energy function of the Hopfield network, and its standard form is as follows:

[0119]

[0120] Equation (12) is applied by referring to the standard writing of the Hopfield network. Among them, x kl represents that the drone executes the flight mission in the building area l in the k-th flight sortie.

[0121] In Equation (12), W ijkl , I ij represent the weight and bias, and the expansion is expressed as:

[0122]

[0123] In Equation (13), δ ij represents the indicator function.

[0124] In this embodiment, the penalty parameters A = B = D = 50 and C = 20 are set; the indicator function When i = j, δ ij takes the value of 1; when i ≠ j, δ ij takes the value of 0.

[0125] S900: Iteratively correct the weight matrix of the energy function until the entire output state tends to be stable, end the iteration, and output the result.

[0126] The specific implementation process of this step is as follows:

[0127] Step 1: Set the penalty parameters A, B, C, D and the maximum number of iterations T max , and randomly initialize the order matrix x;

[0128] Step 2: Calculate the distances between each building area to form the distance order matrix d, and d ij represents the distance from building area i to building area j;

[0129] Step 3: Determine the state transition function U 0 according to the initialized input state U ki (t):

[0130]

[0131] Step 4: Calculate the increment ΔU of the input state:

[0132]

[0133] Step 5: Use the dynamic equation and the first-order Euler method to update the input state and output state of the next moment:

[0134]

[0135] Step 6: Check the current output status x ki Whether the constraints are met, determine whether the iteration is over, otherwise return to step 4 until the number of iterations reaches the set maximum number.

[0136] In this embodiment, the penalty coefficients A=B=D=50, C=20, and the maximum number of iterations T max is 50 times, and the energy function iteration process is shown in Figure 11 (b), when the number of iterations reaches about 30, the output state is close to stable, and the minimum energy loss is 315.6542161747969; combined with the local route, the global route result of the entire measurement area is obtained, such as Figure 13 shown.

[0137] The above description is only a specific implementation method of the present invention, and these descriptions are only for explaining the principle of the present invention, and cannot be interpreted as limiting the protection scope of the present invention in any way. Based on the explanation here, those skilled in the art can associate other specific implementation methods of the present invention without creative work, and these methods will fall within the protection scope of the present invention.

Claims

1. Optimal path planning method for large-scale aerial survey of buildings based on Hopfield network, Its characteristics are: include: The vegetation area is located using near-infrared images covering the entire large-scale target survey area, and the surface topography of the large-scale target survey area is estimated using multi-view aerial images. The vegetation area positioning results and the surface topography estimation results are combined to locate each building area in the entire large-scale target survey area. Extract edge features of any building area, transform the edge features into straight line features, obtain the main direction of the building area according to the direction vector of the straight line features, adjust the flight heading during aerial survey of the building area to be perpendicular to the main direction, and generate a local route of the building area according to the overlap rate requirements; According to the positions of each building area in the large-scale target measurement area, the distance order matrix between building areas is determined. According to the distance order matrix, the shortest path objective function and constraints of global route optimization are set. According to the objective function and constraints, the Hopfield network standard energy function is constructed. The Hopfield network standard energy function is iteratively solved with the goal of minimizing energy loss to obtain the overall route covering all building areas in the entire large-scale target measurement area.

2. The optimal path planning method for large-scale aerial survey of buildings based on Hopfield network as claimed in claim 1, Its characteristics are: The method of estimating the surface topography of a large-scale target survey area by using multi-view aerial images is specifically as follows: Use the drone to take a bird's-eye view of the entire large-scale target survey area from the take-off point, and record the position and posture information of the drone sensor at the moment of shooting; The mobile drone takes a bird's-eye view of the entire large-scale target measurement area from another location, and uses the drone's inertial navigation data combined with the initialization posture information to calculate the posture information at the moment of the second shooting; Match the two images based on the grayscale features, calculate the depth of the matching point pairs, and obtain the estimated three-dimensional coordinates of the ground points; The surface topography of a large-scale target survey area is fitted using the estimated three-dimensional coordinates.

3. The optimal path planning method for large-scale aerial survey of buildings based on Hopfield network as claimed in claim 1, Its characteristics are: The positioning of each building area within the entire large-scale target measurement area is performed by combining the vegetation area positioning result and the surface terrain estimation result, specifically: Determine whether the pixel (x, y) meets the conditions If it meets the conditions, the pixel (x, y) belongs to the building pixel; among them, NDVI(x, y) represents the NDVI value of the pixel (x, y), which is the positioning result of the vegetation area; F'(X, Y) is the estimated value of the surface terrain; t 1 is the gray-scale threshold used to distinguish vegetation, t 2 is the height threshold used to distinguish the surface and non-surface.

4. The optimal path planning method for large-scale aerial survey of buildings based on Hopfield network as claimed in claim 1, Its characteristics are: The edge features of the building area are extracted by using an 8-direction enhanced edge detection operator, and the structure of the 8-direction enhanced edge detection operator is as follows: The operator kernel is denoted as g m , for the pixel (x, y), according to judge whether the pixel (x, y) is an edge pixel; where, t 3 is the threshold for judging whether it is an edge pixel, and N 5×5 (x, y) represents the 5×5 neighborhood pixels of the pixel (x, y).

5. The optimal path planning method for large-scale aerial survey of buildings based on Hopfield network as claimed in claim 1, Its characteristics are: The step of obtaining the main direction of the building area according to the direction vector of the straight line segment feature further comprises: Calculate the direction vector of the straight-line segment feature, specifically: transform the straight-line segment feature into the polar coordinate system, and the polar coordinates are denoted as (θ s , ρ s ), where θ s and ρ s represent the direction angle and length of the straight-line segment feature respectively, and use θ s as the direction vector of the straight-line segment feature; Divide the line segment features into preset different direction intervals according to the direction vector, specifically: evenly divide the angle range [0, π] into several direction intervals; use the formula to transform the direction vector θ s and classify the transformed direction vector into the corresponding direction interval. The main direction of the building area is obtained according to the direction vectors in each direction interval, specifically: the direction vectors in each direction interval are summed to obtain the sum vector, and the direction interval corresponding to the sum vector with the maximum modulus is taken as the main direction of the building area.

6. The optimal path planning method for large-scale building aerial survey based on the Hopfield network as described in claim 1, characterized in that: The objective function is The constraint conditions are 7. The optimal path planning method for large-scale building aerial survey based on the Hopfield network as described in claim 1, characterized in that: The construction of the standard energy function of the Hopfield network according to the objective function and the constraint conditions is specifically as follows: the constraint conditions and the objective function are transformed into penalty terms of the energy function.

8. The optimal path planning method for large-scale building aerial survey based on the Hopfield network as described in claim 7, characterized in that: The constructed standard energy function of the Hopfield network is:

9. The optimal path planning method for large-scale building aerial survey based on the Hopfield network as described in claim 1, characterized in that: The iterative solution of the standard energy function of the Hopfield network with the minimum energy loss as the goal further includes: Step 1: Set penalty parameters A, B, C, D and the maximum number of iterations T max , and randomly initialize the order matrix x; Step 2: Calculate the distances between building areas to form a distance order matrix d, where d ij represents the distance from building area i to building area j; Step 3: Determine the state transition function U 0 according to the initialized input state U ki (t): Step 4: Calculate the increment ΔU of the input state: Step 5: Update the input state and output state of the next moment using the dynamic equation and the first-order Euler method: Step 6: Check whether the current output state xki satisfies the constraints, and determine whether the iteration ends. Otherwise, return to Step 4 until the number of iterations reaches the set maximum number of times; E d represents the standard energy function of the Hopfield network: W ijkl ,I ij represent weights and biases: N represents the total number of building areas, x ij represents that the drone performs the flight mission of building area j in the i-th flight sortie, x kl represents that the drone performs the flight mission of building area l in the k-th flight sortie, d ik represents the distance from the i-th flight area to the k-th flight area. A, B, C, and D are all penalty parameters, δ ij represents the indicator function.

10. The optimal path planning system for large-scale building aerial survey based on the Hopfield network, characterized in that, including: A building area positioning module, which is used to locate the vegetation area by using the near-infrared images covering the entire large-scale target survey area, estimate the surface terrain of the large-scale target survey area by using multi-view aerial images, and combine the vegetation area positioning result and the surface terrain estimation result to locate each building area in the entire large-scale target survey area; A single-area route planning module, which is used to extract the edge features of any building area, transform the edge features into straight-line segment features, obtain the main direction of the building area according to the direction vector of the straight-line segment features, adjust the flight heading during the aerial survey of the building area to be perpendicular to the main direction, and generate the local route of the building area according to the overlap rate requirement; An overall survey area route optimization module, which is used to determine the distance order matrix between building areas according to the positions of each building area in the large-scale target survey area, set the shortest path objective function and constraint conditions for global route optimization according to the distance order matrix, construct the standard energy function of the Hopfield network according to the objective function and the constraint conditions, and iteratively solve the standard energy function of the Hopfield network with the minimum energy loss as the goal to obtain the overall route covering all building areas in the entire large-scale target survey area.

Images