Intelligent route planning method and system for expressway unmanned aerial vehicle

Abstract

The invention relates to the technical field of expressway unmanned aerial vehicle route planning, in particular to an intelligent route planning method and system for an expressway unmanned aerial vehicle, and the method specifically comprises the steps: planning unmanned aerial vehicle waypoints on an expressway based on a target three-dimensional model, and obtaining a waypoint set of the expressway; historical route data of the expressway are obtained, a deep learning algorithm is used for analyzing the historical route data, safe flight path features are extracted, a route safety evaluation model is constructed, and the route safety evaluation model is used for evaluating the route safety level of the route of the unmanned aerial vehicle; and determining route planning parameters according to the unmanned aerial vehicle task type, and determining an unmanned aerial vehicle route through the waypoint set, the route safety evaluation model and the waypoint planning parameters based on a multi-objective optimization algorithm. According to the invention, the safety and efficiency of the unmanned aerial vehicle in highway inspection and monitoring tasks are improved, and automatic planning and dynamic adjustment of the route of the unmanned aerial vehicle based on real-time data analysis are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of highway unmanned aerial vehicle route planning, and in particular to an intelligent route planning method and system for a highway unmanned aerial vehicle. Background Art

[0002] With the rapid development of drone technology and the intensive introduction of national and provincial policy documents to promote the application of drones in the highway industry, drones are increasingly used in business scenarios such as dynamic real-time monitoring of highways, road surface and bridge and tunnel inspections, and emergency rescue, becoming an important means to promote the high-quality development of highways.

[0003] When drones are used to perform tasks on highways, in order to ensure the efficiency, quality and safety of the drone mission execution, it is necessary to plan the drone route so that the route passes through the preset waypoints. However, due to the diversity of highway business scenarios and the complex on-site environment of highways, with structures such as bridges, tunnels, and slopes, it is very labor-intensive to plan routes according to different drone mission requirements and has high requirements for operators. In addition, traditional route planning methods cannot comprehensively consider the requirements of highway restricted flight areas, which brings safety risks.

[0004] The existing methods for planning routes for UAVs on highways mainly fall into two categories: on-site planning and GIS map-based planning. Among them, the on-site planning method is that operators operate UAVs on the highway site to plan routes, which is costly and requires continuous adjustment of parameters such as flight routes and flight altitudes during the flight to select appropriate target waypoints. This is very time-consuming and cumbersome, and also requires high requirements for operators, and is often operated by professionals with UAV licenses. GIS map-based planning is to manually select waypoints on the highway GIS map according to the requirements of the UAV mission, and set parameters such as the altitude and speed of the waypoints. Since different UAV missions have different route requirements, and the planning of routes requires comprehensive consideration of realistic factors such as the spatial position, obstacles, and endurance of the UAV during flight, it is impossible to ensure that the planned route meets the actual use needs. In addition, the GIS map lacks information on restricted flight areas on highways, and it is impossible to ensure that the planned route meets the requirements for safe flight, which poses a hidden danger to the safety of highway operations. Summary of the invention

[0005] The object of the present invention is to provide an intelligent route planning method and system for a highway UAV to solve at least one of the above-mentioned prior art problems.

[0006] In a first aspect, the present invention provides an intelligent route planning method for a highway drone, the method specifically comprising:

[0007] Using laser radar scanning technology to perform three-dimensional scanning of the highway to obtain three-dimensional point cloud data, segmenting the three-dimensional point cloud data using a point cloud segmentation algorithm to determine various structures of the highway, and constructing a three-dimensional terrain model of the highway according to the type and geometric characteristics of each structure;

[0008] Acquire flight restriction area data of the expressway, construct a three-dimensional flight restriction model based on the flight restriction area data, and generate a target three-dimensional model according to the three-dimensional terrain model and the three-dimensional flight restriction model;

[0009] Planning UAV waypoints on the highway based on the target three-dimensional model to obtain a waypoint set of the highway;

[0010] Acquire historical route data of the expressway, analyze the historical route data using a deep learning algorithm, extract safe flight path features, and construct a route safety assessment model, wherein the route safety assessment model is used to assess the route safety level of the UAV route;

[0011] Determine route planning parameters according to the type of UAV mission, and determine the UAV route through the waypoint set, the route safety assessment model and the waypoint planning parameters based on a multi-objective optimization algorithm;

[0012] A convolutional neural network is used to perform real-time analysis of aerial images to identify and determine the location and movement trend of obstacles, and the UAV route is dynamically adjusted according to the location and movement trend of the obstacles.

[0013] In a second aspect, the present invention provides an intelligent route planning system for a highway drone, the system specifically comprising:

[0014] The first planning module is used to perform three-dimensional scanning of the highway using laser radar scanning technology to obtain three-dimensional point cloud data, segment the three-dimensional point cloud data using a point cloud segmentation algorithm, determine various structures of the highway, and construct a three-dimensional terrain model of the highway according to the type and geometric characteristics of each structure;

[0015] A second planning module is used to obtain flight restriction area data of the expressway, construct a three-dimensional flight restriction model based on the flight restriction area data, and generate a target three-dimensional model according to the three-dimensional terrain model and the three-dimensional flight restriction model;

[0016] A third planning module is used to plan UAV waypoints on the highway based on the target three-dimensional model to obtain a waypoint set of the highway;

[0017] The fourth planning module is used to obtain the historical route data of the expressway, analyze the historical route data using a deep learning algorithm, extract the safe flight path features, and construct a route safety assessment model, wherein the route safety assessment model is used to assess the route safety level of the UAV route;

[0018] A fifth planning module, used to determine the route planning parameters according to the UAV mission type, and determine the UAV route based on the waypoint set, the route safety assessment model and the waypoint planning parameters based on a multi-objective optimization algorithm;

[0019] The sixth planning module is used to use a convolutional neural network to perform real-time analysis of aerial images, identify and determine the location and movement trend of obstacles, and dynamically adjust the UAV route according to the location and movement trend of the obstacles.

[0020] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor and a computer program stored in the memory. When the computer program is executed on the processor, an intelligent route planning method for a highway drone as described in any of the above methods is implemented.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for intelligent route planning for a highway drone as described in any one of the above methods is implemented.

[0022] Compared with the prior art, the present invention has at least one of the following technical effects:

[0023] 1. It improves the safety and efficiency of UAVs in highway inspection and monitoring tasks, realizes automatic planning and dynamic adjustment of UAV routes based on real-time data analysis, and improves the operational safety and efficiency of UAVs in complex environments.

[0024] 2. Combined with the restricted flight area data of the highway, a three-dimensional restricted flight model was constructed, and it was integrated with the three-dimensional terrain model to generate a target three-dimensional model, which can fully reflect the flight restricted areas on the highway, ensure that the planned drone routes strictly comply with safe flight regulations, avoid conflicts with no-fly zones, danger zones, etc., and significantly improve flight safety.

[0025] 3. Through deep learning algorithms, historical route data is analyzed to extract the characteristics of safe flight paths, and a route safety assessment model is constructed. This can assess the safety level of drone routes and provide important reference for route planning.

[0026] 4. Route optimization based on historical experience makes the planned routes more in line with actual flight needs, improving flight efficiency and reliability.

[0027] 5. Determine the route planning parameters according to the UAV mission type, comprehensively consider the waypoint set, route safety assessment model and waypoint planning parameters, and determine the final UAV route. It can find the optimal or suboptimal flight path under the premise of meeting multiple constraints, thereby improving the scientificity and rationality of route planning.

[0028] 6. It can dynamically adjust the UAV route to avoid sudden obstacles, ensure flight safety, enhance the UAV's ability to cope with complex environments, and improve the flexibility and robustness of flight. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 It is a flowchart of an intelligent route planning method for a highway UAV provided by the first embodiment of the present invention;

[0031] Figure 2 It is a flowchart of an intelligent route planning method for a highway UAV provided by a second embodiment of the present invention;

[0032] Figure 3 Schematic diagram of the flight sequence of the UAV at the uplink waypoint in the second embodiment of the present invention;

[0033] Figure 4 Schematic diagram of the flight sequence of the UAV at the downlink waypoint in the second embodiment of the present invention;

[0034] Figure 5 A schematic diagram of obtaining the best waypoint when the waypoint directions of the first waypoint and the last waypoint are the same in the second embodiment of the present invention;

[0035] Figure 6 It is a schematic diagram of obtaining the best waypoint when the first waypoint is an uplink waypoint and the last waypoint is a downlink waypoint in the second embodiment of the present invention;

[0036] Figure 7 It is a schematic diagram of obtaining the best waypoint when the first waypoint is a downlink waypoint and the last waypoint is an uplink waypoint in the second embodiment of the present invention;

[0037] Figure 8 A schematic diagram of the structure of an intelligent route planning system for a highway UAV provided by one embodiment of the present invention;

[0038] Fig. 9It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0040] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0041] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0042] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0043] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0044] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0045] In the embodiment of the present application, the execution subject of the process includes a terminal device, which includes but is not limited to: a server, a computer, a smart phone, a tablet computer, and other devices capable of executing the method disclosed in the present application. Figure 1 A schematic flow chart of an intelligent route planning method for a highway drone disclosed in the first embodiment of the present invention is shown, and the details are as follows:

[0046] S101, use laser radar scanning technology to perform three-dimensional scanning of the highway to obtain three-dimensional point cloud data, segment the three-dimensional point cloud data through a point cloud segmentation algorithm, determine various structures of the highway, and construct a three-dimensional terrain model of the highway according to the type and geometric characteristics of each structure.

[0047] In this embodiment, a laser radar scanning technology is used to perform a three-dimensional scan of the highway to obtain the three-dimensional point cloud data of the highway. The three-dimensional point cloud data is segmented by a point cloud segmentation algorithm based on region growth to obtain the point cloud data of various structures such as the road surface, shoulder, guardrail, isolation belt, etc. of the highway. According to the geometric features of the point cloud data of each structure obtained by segmentation, a three-dimensional model of each structure is constructed by a triangular meshing method. The constructed three-dimensional models of each structure are spliced ​​to obtain a complete three-dimensional terrain model of the highway. The three-dimensional point cloud data obtained by three-dimensional scanning of the highway are further optimized by a point cloud segmentation algorithm based on graph cuts to improve the segmentation accuracy of the point cloud data of each structure. According to the optimized point cloud data of each structure, a high-precision three-dimensional model of each structure is reconstructed by a Poisson reconstruction algorithm. The reconstructed high-precision three-dimensional models of each structure are spliced ​​to obtain a fine three-dimensional terrain model of the highway. According to the obtained fine three-dimensional terrain model of the highway, the three-dimensional terrain model is texture mapped by a texture mapping technology to obtain a three-dimensional terrain model of the highway with real texture information. The illumination effect of the three-dimensional terrain model is calculated by an illumination calculation technology to simulate the real scene of the highway under real illumination conditions. The calculated lighting effects are applied to the 3D terrain model of the highway with real texture information to generate a realistic 3D virtual scene of the highway.

[0048] In some embodiments, in the above step S101, the types of the structures include bridge structures, tunnel structures and slope structures; and constructing the three-dimensional terrain model of the highway according to the types and geometric features of each structure specifically includes:

[0049] The three-dimensional point cloud data of the bridge structure is converted into a triangular mesh surface by using a triangular mesh method, and the triangular mesh surface is processed by a surface reconstruction technology to obtain a first three-dimensional terrain model of the bridge structure;

[0050] Using a cylindrical surface fitting method to fit the three-dimensional point cloud data of the tunnel structure into a plurality of cylindrical surfaces, and by splicing the plurality of cylindrical surfaces, a second three-dimensional terrain model of the tunnel structure is obtained;

[0051] The three-dimensional point cloud data of the slope structure is converted into a polygonal mesh surface by using a polygonal meshing method, and the polygonal mesh surface is processed by a surface reconstruction technology to obtain a third three-dimensional terrain model of the slope structure.

[0052] In this embodiment, according to the three-dimensional point cloud data of the bridge structure, a triangular meshing method is used to convert the point cloud data into a triangular mesh surface to obtain an initial triangular mesh model of the bridge structure. The initial triangular mesh model of the bridge structure obtained in the previous step is processed by surface reconstruction technology, the topological structure and geometric shape of the triangular mesh surface are optimized, and a more refined and smooth three-dimensional mesh surface model of the bridge structure is obtained. According to the reconstructed three-dimensional mesh surface model of the bridge structure, a mesh simplification algorithm is used to reduce the number of facets of the triangular mesh surface under the premise of ensuring the key features of the bridge structure, and a simplified three-dimensional mesh model of the bridge structure is obtained to improve the efficiency of subsequent processing. The simplified three-dimensional mesh model of the bridge structure is locally refined by a mesh subdivision algorithm, and the mesh density is increased in key parts of the bridge structure such as piers, bridge towers, bridge decks, etc., to obtain a three-dimensional model of the bridge structure with richer details. Texture mapping technology is used to map the actual texture image of the bridge structure onto the three-dimensional mesh model to obtain a three-dimensional model of the bridge structure with a real texture effect, thereby improving the visual realism of the model. According to the actual material properties of the bridge structure, the material of the 3D model is edited, and the material parameters of different parts of the bridge, such as color, reflectivity, roughness, etc., are set to obtain a more realistic 3D model of the bridge structure. Through the lighting and rendering settings, the 3D model of the bridge structure is rendered, and appropriate light sources, shadows and environmental effects are set to obtain a 3D rendering of the bridge structure with realistic light and shadow effects, thereby improving the visual quality of the model. According to the actual size and proportion of the bridge structure, the 3D model is dimensioned and measured to obtain the key dimensional parameters of the bridge structure, such as the span length, pier height, bridge deck width, etc., to provide data support for subsequent bridge maintenance and management. Finally, based on the processing results of the above steps, a high-precision, high-realism 3D terrain model of the bridge structure containing rich details and attribute information is obtained.

[0053] According to the three-dimensional point cloud data of the tunnel structure, a cylindrical surface fitting method is used to fit the three-dimensional point cloud data of the tunnel structure into several cylindrical surfaces, and the position and shape information of each cylindrical surface is obtained through the cylindrical surface parameters obtained by fitting. According to the position and shape information of each cylindrical surface obtained, several cylindrical surfaces are spliced ​​through space transformation and coordinate system one to obtain a complete cylindrical surface splicing model of the tunnel structure. The cylindrical surface splicing model of the tunnel structure is rendered using rendering technologies such as ray tracing to obtain a second three-dimensional terrain model of the tunnel structure, which can realistically reflect the three-dimensional geometric shape and spatial position relationship of the tunnel.

[0054] According to the large amount of scattered point cloud data obtained by scanning the slope structure with a three-dimensional laser scanner, the point cloud data is converted into a coordinate system using a spatial coordinate transformation method, and then converted into a specified spatial rectangular coordinate system to obtain the original three-dimensional point cloud data in the unified coordinate system of the slope structure. The noise points and outliers that obviously deviate from the slope surface are removed from the point cloud data by removing noise and filtering outliers, and the three-dimensional point cloud data of the slope structure after noise removal is obtained. The point cloud data after noise removal is processed into polygonal meshes using a triangular meshing method, and the point cloud is fitted by triangular plane elements. A large number of triangular meshes are used to approximate the surface morphology of the slope to obtain polygonal mesh surface data that can reflect the surface morphological characteristics of the slope structure. According to the generated polygonal mesh surface, the surface reconstruction technology is used to optimize the polygonal mesh surface through methods such as B-spline surface fitting and surface smoothing to eliminate the noise and irregularity of the surface, and obtain a smooth and realistic three-dimensional surface model of the slope structure.

[0055] S102, obtaining flight restriction area data of the expressway, constructing a three-dimensional flight restriction model according to the flight restriction area data, and generating a target three-dimensional model according to the three-dimensional terrain model and the three-dimensional flight restriction model.

[0056] In this embodiment, the restricted flight area data of the expressway is obtained, and the restricted flight area data is converted into three-dimensional spatial coordinate point cloud data by a spatial coordinate conversion algorithm. According to the converted three-dimensional spatial coordinate point cloud data, a three-dimensional restricted flight model is constructed by a three-dimensional Alpha shape algorithm to obtain a three-dimensional geometric model representing the restricted flight area. The terrain data of the expressway is obtained, and the terrain data is converted into three-dimensional spatial coordinate point cloud data by a spatial coordinate conversion algorithm. According to the converted three-dimensional spatial coordinate point cloud data, a three-dimensional terrain model is constructed by a Thiessen polygon algorithm to obtain a three-dimensional geometric model representing the terrain undulations. The constructed three-dimensional restricted flight model and the three-dimensional terrain model are unified and registered in the spatial coordinate system so that the two models are in the same three-dimensional spatial coordinate system. A three-dimensional Boolean operation is performed on the registered three-dimensional restricted flight model and the three-dimensional terrain model, and the part of the three-dimensional terrain model located in the restricted flight area is removed by subtracting the three-dimensional models to obtain a model of the flyable area in the three-dimensional terrain model. The three-dimensional restricted flight model and the flyable area model are subjected to a three-dimensional model merging operation, and a complete target three-dimensional model data is generated by a three-dimensional mesh splicing algorithm, and the model contains the three-dimensional geometric information of the restricted flight area and the flyable area. The generated target 3D model is simplified and optimized, and a 3D mesh simplification algorithm is used to reduce the number of facets and vertices of the model while ensuring a certain precision loss, so as to obtain a lightweight 3D model with smaller data volume and higher rendering efficiency. The simplified target 3D model data is format converted and compressed, and converted into a universal 3D data exchange format through a 3D model format conversion algorithm, and a lossless compression algorithm is used to compress the 3D model data, reducing the storage size of the 3D model file, which is convenient for the transmission, loading and rendering of the 3D scene of the highway.

[0057] In some embodiments, in the above step S102, constructing a three-dimensional flight restriction model using the flight restriction area data, and generating a target three-dimensional model according to the three-dimensional terrain model and the three-dimensional flight restriction model specifically includes:

[0058] Determine the vertical range, horizontal range and flight restriction requirements at different altitudes of the restricted flight area according to the restricted flight area data;

[0059] Determine the road section range affected by the restricted flight area of ​​the expressway according to the vertical range and the horizontal range;

[0060] Determine the flight altitude limit for each location within the route section according to the flight restriction requirements at different altitudes;

[0061] Constructing a three-dimensional flight restriction model according to the road section range and the flight altitude restrictions of various locations within the road section range;

[0062] After the spatial coordinate systems of the three-dimensional terrain model and the three-dimensional flight restriction model are unified and registered, the three-dimensional terrain model and the three-dimensional flight restriction model are subjected to a three-dimensional model merging operation through a three-dimensional grid stitching algorithm to generate a target three-dimensional model.

[0063] In this embodiment, according to the restricted flight area data, a spatial coordinate analysis algorithm is used to obtain the vertical height range and horizontal longitude and latitude range of the restricted flight area, and the specific flight restriction requirements of each altitude layer are obtained by analyzing the restricted flight rules at different altitudes. According to the obtained vertical height range and horizontal longitude and latitude range of the restricted flight area, a spatial superposition analysis algorithm is used to superimpose the restricted flight area with the highway section to obtain the highway section range affected by the restricted flight area. According to the obtained flight restriction requirements of each altitude layer, a spatial interpolation algorithm is used to calculate the flight height limit for each position within the obtained affected section range to obtain the specific flight height limit value for each position within the section range. According to the obtained affected section range and the obtained flight height limit value for each position within the section range, a three-dimensional modeling algorithm is used to combine the two-dimensional section range with the flight height limit value to construct a three-dimensional flight restriction model covering the section range.

[0064] According to the three-dimensional terrain model and the three-dimensional flight restriction model, the coordinate systems of the two models are unified by using the unification and registration method of the spatial coordinate system, so as to obtain the three-dimensional terrain model and the three-dimensional flight restriction model under the unified coordinate system. The mesh data of the three-dimensional terrain model and the three-dimensional flight restriction model under the unified coordinate system are obtained by using the three-dimensional mesh splicing algorithm, and the meshes of the two models are spliced ​​to determine the spliced ​​mesh model, which contains the geometric information of the three-dimensional terrain and the three-dimensional flight restriction area. According to the spliced ​​mesh model, the three-dimensional terrain model and the three-dimensional flight restriction model are merged by using the three-dimensional model merging algorithm to obtain the merged three-dimensional model data, which contains both the terrain information and the flight restriction area information. By performing geometric optimization and texture mapping on the merged three-dimensional model, the target three-dimensional model is finally generated, which can truly reflect the three-dimensional spatial information of the terrain environment and the flight restriction area, and provide data support for subsequent flight planning and management.

[0065] S103: Planning UAV waypoints on the highway based on the target three-dimensional model to obtain a waypoint set of the highway.

[0066] In this embodiment, the target three-dimensional model is transformed into spatial coordinates, and the coordinate system of the target three-dimensional model is aligned with the coordinate system of the highway to obtain the aligned target three-dimensional model. According to the aligned target three-dimensional model, the highway is spatially divided by a three-dimensional space division algorithm to obtain a divided highway spatial grid model. According to the divided highway spatial grid model, a three-dimensional space path planning algorithm is used to plan the flight path of the UAV on the highway to obtain the waypoint flight path of the UAV. According to the waypoint flight path of the UAV, a three-dimensional space sampling algorithm is used to perform waypoint sampling on the waypoint flight path to obtain a sampled waypoint set. By performing coordinate transformation on the sampled waypoint set, the coordinate system of the sampled waypoint set is aligned with the global coordinate system to obtain the waypoint set of the highway. If the waypoint set of the highway does not meet the preset conditions, the division granularity of the three-dimensional space division algorithm and the planning parameters of the three-dimensional space path planning algorithm are adjusted, and the above steps are re-executed until the waypoint set of the highway meets the preset conditions. By optimizing the waypoint set of the highway, a three-dimensional space curve fitting algorithm is used to fit the waypoint set to obtain a fitted waypoint flight curve. According to the fitted waypoint flight curve, a kinematic model is used to calculate the flight attitude of the UAV to obtain the waypoint flight attitude data of the UAV, and the waypoint flight attitude data is associated with the waypoint set of the highway to obtain the final waypoint flight data.

[0067] In some embodiments, in the above step S103, planning drone waypoints on the highway based on the target three-dimensional model to obtain a waypoint set of the highway specifically includes:

[0068] Based on the target three-dimensional model, the expressway is spatially divided using a three-dimensional space division algorithm to obtain a spatial grid model of the expressway;

[0069] According to the highway spatial grid model, a three-dimensional spatial path planning algorithm is used to plan a flight path of the UAV on the highway to obtain a waypoint flight path of the UAV;

[0070] A three-dimensional space sampling algorithm is used to perform waypoint sampling on the waypoint flight path to obtain a sampled waypoint set.

[0071] In this embodiment, according to the target three-dimensional model, the octree space partitioning algorithm is used to recursively divide the three-dimensional space of the highway into equal-sized cubic grid units to obtain the highway space grid model, and each grid unit contains information such as roads, terrain, facilities, etc. in its spatial range. According to the highway space grid model, a three-dimensional space path planning algorithm is used to obtain the flyable space grid on the highway. By analyzing the properties such as the position and height of the flyable space grid, the take-off position and target position of the drone are determined. According to the take-off position and the target position, the A* search algorithm is used to search the optimal flight path in the three-dimensional space grid to obtain a rough flight path composed of a series of space grids. The Bezier curve smoothing algorithm is used to smooth the rough flight path to obtain a smooth waypoint flight path. By performing a safety assessment on the smooth waypoint flight path, it is determined whether the flight path meets the performance constraints and airspace control requirements of the drone. According to the safety assessment results, the flight path that does not meet the requirements is adjusted and optimized to obtain the final executable drone waypoint flight path. According to the initial data of the waypoint flight path, a three-dimensional space sampling algorithm is used to obtain a three-dimensional coordinate point set of the entire flight path. Through these coordinate point sets, a complete waypoint set is obtained. Using this waypoint set, the specific position and relative altitude of each waypoint are determined to ensure that each waypoint can accurately represent the actual position of the aircraft in space.

[0072] S104, obtaining historical route data of the highway, analyzing the historical route data using a deep learning algorithm, extracting safe flight path features, and constructing a route safety assessment model, wherein the route safety assessment model is used to assess the route safety level of the drone route.

[0073] In this embodiment, the historical route data of the expressway is obtained, and the historical route data is analyzed using a long short-term memory neural network algorithm. Through deep learning analysis of the historical route data, the safe flight path features are extracted, including characteristic parameters such as flight altitude, flight speed, flight direction, and flight distance. According to the extracted safe flight path features, a route safety assessment model is constructed, and the route safety assessment model is trained using a support vector machine algorithm to obtain a trained route safety assessment model. The route safety assessment model judges the safety level of the route by inputting the flight parameters of the drone route. If the flight parameters of the route meet the safe flight path characteristics, the route is determined to be a safe route. If the flight parameters of the route do not meet the safe flight path characteristics, the route is determined to be a dangerous route. The route safety level of the drone route is determined by the evaluation results of the route safety assessment model, and the route safety level is divided into five levels: safe, relatively safe, general, relatively dangerous, and dangerous. According to the route safety level evaluated by the route safety assessment model, the flight path of the drone is adjusted and optimized. For routes evaluated as dangerous levels, the flight path needs to be replanned, and routes with high safety levels are selected as the flight paths of the drone. By evaluating and optimizing the safety of UAV flight paths, safe UAV routes can be obtained, thus ensuring the safety of UAV flight.

[0074] In some embodiments, in the above step S104, the use of a deep learning algorithm to analyze the historical route data, extract safe flight path features, and construct a route safety assessment model specifically includes:

[0075] Using a long short-term memory neural network to extract features from the historical route data to obtain safe flight path features;

[0076] According to the safe flight path characteristics, a first route safety assessment model is obtained by performing classification training through a support vector machine, where the first route safety assessment model is used to determine whether the route is safe;

[0077] Acquiring safety assessment index data according to the historical route data, the safety assessment index data including collision risk data, meteorological condition data and electromagnetic interference data;

[0078] Use fuzzy comprehensive evaluation method to quantify and score each safety assessment index data to obtain the evaluation score of each safety assessment index data;

[0079] The weighted average method is used to sum the evaluation scores of each safety evaluation index data to obtain the comprehensive safety score of the route;

[0080] According to the comprehensive safety score of the route, the route is divided into safety levels by using a cluster analysis method to obtain a set of routes with different safety levels;

[0081] Taking the route set of different safety levels and each safety assessment index data as input, a second route safety assessment model is constructed through a decision tree algorithm, and the second route safety assessment model is used to assess the safety level of the route.

[0082] In this embodiment, based on the historical route data, a long short-term memory neural network is used to extract features from the historical route data, and flight parameters such as the speed, altitude, and heading of the route are obtained as input. The feature change trend of the route in the time series is extracted through the multi-layer network structure and gate control unit of the long short-term memory neural network, and a feature vector reflecting the safety of the route is obtained as the safe flight path feature. Based on the obtained safe flight path feature, a support vector machine algorithm is used to train the classification model. The safe flight path feature is used as the feature vector of the training sample, and the corresponding route safety is used as the sample label. Samples of different categories are segmented by finding the maximum margin hyperplane to obtain a support vector machine classification model that can distinguish the route safety, that is, the first route safety assessment model. Using new route data as input, the safe flight path feature of the route is extracted through the long short-term memory neural network, and the extracted feature is input into the first route safety assessment model. The support vector machine classification model determines whether the route belongs to a safe route based on the positional relationship between the feature vector and the classification hyperplane, thereby evaluating the safety of the route and obtaining a route safety assessment result.

[0083] According to the historical route data, data collection and preprocessing technology are used to obtain safety assessment index data such as collision risk data, meteorological conditions data and electromagnetic interference data. The fuzzy comprehensive evaluation method is used to quantify and score each safety assessment index data by establishing an evaluation index set, a comment set and a fuzzy relationship matrix, and obtain the evaluation score of each safety assessment index data. According to the importance of each safety assessment index, the weighted average method is used to set the weight coefficient and perform weighted summation on the evaluation score of each safety assessment index data to obtain a score reflecting the comprehensive safety of the route. According to the comprehensive safety score of the route, the cluster analysis method is used to calculate the distance between samples and group them according to the distance size, and the route is divided into safety levels to obtain a set of routes with different safety levels. The set of routes with different safety levels and each safety assessment index data are used as the input of the decision tree algorithm. By recursively selecting the optimal features, a second route safety assessment model is constructed to evaluate the safety level of the new route. Using the new route data as input, the second route safety assessment model is used to judge the safety level of the new route based on each safety assessment index data, and obtain the evaluation result of route safety.

[0084] S105, determining route planning parameters according to the UAV mission type, and determining the UAV route based on the waypoint set, the route safety assessment model and the waypoint planning parameters based on a multi-objective optimization algorithm.

[0085] In this embodiment, the multi-objective optimization algorithm is an ant colony algorithm. The route planning parameters are determined according to the type of UAV mission, including parameters such as route length, number of waypoints, and distance between waypoints. The ant colony algorithm is used to plan the UAV route, and the UAV mission area is discretized into a set of waypoints, each of which contains information such as location coordinates and elevation. By constructing a route safety assessment model, the safety assessment value of each waypoint is calculated by comprehensively considering factors such as terrain undulation, building density, and electromagnetic interference intensity. According to the waypoint set, the route safety assessment model, and the route planning parameters, the ant colony algorithm is used for route planning. Initialize a number of ants, each ant starts from the starting waypoint, and selects the next waypoint through the state transition rule until it reaches the destination waypoint. During the ant movement, the selection probability of each waypoint is calculated according to the pheromone concentration and the heuristic factor, and the roulette method is used to select the next waypoint. After the ant reaches the destination, the fitness value of the route is calculated according to the route length and the waypoint safety assessment value. After all ants complete one iteration, the pheromone is updated, and the pheromone concentration is proportional to the route fitness value and inversely proportional to the route length. Repeat the above process until the maximum number of iterations is reached or the termination condition is met. During the iteration process, the optimal route and the global optimal route of each iteration are recorded. When the algorithm terminates, the global optimal route is used as the route planning result of the drone. According to the planned route, a waypoint sequence and route coordinates are generated as the reference path for the drone flight.

[0086] Exemplarily, the route planning parameters are determined according to the type of UAV mission. For example, for farmland mapping missions, the route length can be set to 5km, the number of waypoints is 100, and the distance between waypoints is 50m. The ant colony algorithm is used to plan the UAV route, and the mission area is divided into 50m×50m grids. The center point of each grid is used as a waypoint, which contains latitude and longitude coordinates and elevation information. By constructing a route safety assessment model, the safety assessment value of each waypoint is calculated by comprehensively considering factors such as terrain undulation, building density, and electromagnetic interference intensity. For example, for mountainous areas with large terrain undulations, the waypoint safety assessment value can be set to 6, and for urban areas with dense buildings, the waypoint safety assessment value can be set to 4. In the ant colony algorithm, 50 ants are initialized, and each ant starts from the starting waypoint and selects the next waypoint according to the state transition rule, where the pheromone concentration and heuristic factor account for 7 and 3 weights respectively, and the roulette method is used to select the next waypoint. After the ant reaches the destination, the fitness value is calculated based on the route length and the average safety assessment value of the waypoints. For example, if the route length is 8 km and the average safety assessment value is 8, the fitness value can be set to 8 / 8=167. After each iteration, the pheromone concentration is updated according to the fitness value. The update formula is ρ(t+1)=(1-ρ)ρ(t)+Δρ, where ρ is the pheromone volatility coefficient, which is 1, and Δρ is the pheromone increment, which is proportional to the fitness value. The maximum number of iterations of the algorithm is set to 100. When the optimal route is no longer improved after 10 consecutive iterations, the algorithm is considered to have converged and the iteration process is terminated. The final global optimal route is the route planning result of the drone. The waypoint sequence and route coordinates are generated and uploaded to the drone flight control system to guide the drone to perform tasks according to the planned route.

[0087] In some embodiments, in the above step S105, determining the route planning parameters according to the drone mission type, and determining the drone route based on the waypoint set, the route safety assessment model and the waypoint planning parameters based on a multi-objective optimization algorithm specifically include:

[0088] By constructing a mapping relationship between the UAV mission type and the route planning parameters, the route planning parameters under different mission types are obtained, wherein the route planning parameters include flight altitude, speed and heading angle;

[0089] According to the importance of the UAV mission type, the weight of each route planning parameter is calculated using the hierarchical analysis method to obtain the weight vector of route planning constraints;

[0090] Taking route length, flight time and flight energy consumption as optimization targets, taking the waypoint set, the evaluation result of the route safety evaluation model and the route planning constraint weight vector as constraint conditions, adopting a multi-objective optimization algorithm to solve the route planning, and obtaining a Pareto optimal solution set;

[0091] The optimal route that meets the mission requirements is selected from the Pareto optimal solution set as the planning result to determine the UAV route.

[0092] In this embodiment, according to different mission types of UAVs, such as reconnaissance, cruising, mapping, etc., an expert knowledge base method is used to obtain route planning parameters corresponding to each type of mission, including flight altitude, speed and heading angle, and obtain a mapping relationship between the mission type and the planning parameters. Through the hierarchical analysis method, according to the importance of the UAV mission, such as the reconnaissance mission has a high requirement for the timeliness of target information, the weight of each route planning parameter is determined to obtain a route planning constraint weight vector reflecting the mission requirements.

[0093] According to the given waypoint set, the location coordinate information of each waypoint is obtained, and the Euclidean distance between adjacent waypoints is calculated to obtain the route length as one of the optimization goals; the flight time is calculated according to the flight speed of the UAV and the distance between the waypoints, and the flight time is used as one of the optimization goals; according to the energy consumption model of the UAV, combined with parameters such as the distance between the waypoints and the flight altitude, the flight energy consumption is calculated, and the flight energy consumption is used as one of the optimization goals; the route safety assessment model is used to conduct a safety assessment on the candidate routes, and the safety score of each route is obtained, and the safety score is used as one of the constraints; according to the mission requirements, the route planning constraints are set Weight vector, take the weight vector as one of the constraints; use multi-objective optimization algorithm, such as NSGA-II, etc., with route length, flight time, and flight energy consumption as optimization targets, and waypoint set, route safety assessment results, and route planning constraint weight vector as constraints, to solve route planning and obtain the Pareto optimal solution set. From the Pareto optimal solution set, according to the mission requirements, select the optimal route that meets the requirements as the final route planning result. According to the waypoint sequence and position coordinates of the optimal route, determine the flight path of the UAV, generate route planning instructions, transmit them to the UAV flight control system, and guide the UAV to perform the mission according to the planned route.

[0094] S106, using a convolutional neural network to perform real-time analysis on the aerial image, identify and determine the location and movement trend of the obstacle, and dynamically adjust the route of the drone according to the location and movement trend of the obstacle.

[0095] In this embodiment, a convolutional neural network is used to perform real-time analysis of aerial images to identify and determine the location and movement trend of obstacles. By acquiring aerial images, the convolutional neural network identifies obstacles in the image and obtains the location information of the obstacles. According to the location information of the obstacle, the movement trend of the obstacle is analyzed to determine the possible future location of the obstacle. After obtaining the movement trend of the obstacle, the location and movement trend of the obstacle are compared with the current flight path information of the drone to determine whether the route of the drone needs to be adjusted. If the predicted position of the obstacle intersects with the current route of the drone, the route of the drone is adjusted to avoid the predicted position of the obstacle. With the adjusted route information, the drone continues to perform the flight mission, while monitoring the position changes of the obstacles in real time to ensure continuous updating of the route and safe flight.

[0096] In some embodiments, in the above step S106, dynamically adjusting the route of the drone according to the location and movement trend of the obstacle specifically includes:

[0097] Acquire the current drone position data, fuse the position and movement trend of the obstacle with the current drone position data, use the Kalman algorithm to recursively estimate the drone's position and speed, and obtain the drone position estimation result;

[0098] The drone position estimation result is used as a constraint condition to optimize the drone route in real time.

[0099] In this embodiment, a convolutional neural network is used to analyze the drone aerial images in real time, and the ground obstacles and dynamic targets in the image are identified by the convolutional neural network model. The convolutional neural network model adopts the classic VGG16 network structure, is pre-trained on the ImageNet data set, and then fine-tuned on the drone aerial image data set to obtain a convolutional neural network model for obstacle recognition. The obstacle information identified by the convolutional neural network is fused with the drone position data obtained by the airborne inertial navigation system, and the Kalman filter algorithm is used to accurately estimate the position of the drone. The Kalman filter algorithm recursively estimates the position and speed of the drone by establishing a drone kinematic model, combining inertial navigation data and obstacle position information, and obtains an accurate drone position estimation result. According to the obstacle information identified by the convolutional neural network and the drone position estimated by the Kalman filter algorithm, the route dynamic adjustment algorithm is used to optimize the pre-planned optimal route in real time. The route dynamic adjustment algorithm is based on the A* search algorithm, which uses obstacle information and drone position as constraints to search for local safe flight paths in the state space. If the identified obstacle is on the pre-planned route, the fast replanning algorithm is triggered to calculate the local path to avoid the obstacle. The fast replanning algorithm uses the RRT* algorithm, taking the current position of the drone as the starting point and the next waypoint on the scheduled route as the target point, to quickly search for a feasible safe path in the local area to obtain a local flight path that avoids obstacles. According to the local safe flight path obtained by the fast replanning algorithm, the pre-planned optimal route is locally corrected, and the local path is spliced ​​with the original route to obtain an updated global route. The updated route avoids the identified obstacles and ensures the flight safety of the drone. By continuously executing the above steps, the real-time dynamic optimization of the drone's flight route is achieved. The convolutional neural network continuously analyzes aerial images and identifies new obstacles; the Kalman filter algorithm continuously estimates the drone's position; the route dynamic adjustment algorithm continuously optimizes the global route based on the latest obstacle information and drone position, so that the drone can complete its flight mission safely and efficiently.

[0100] Reference Figure 2 , Figure 2 A flowchart of an intelligent route planning method for a highway drone according to a second embodiment of the present invention is disclosed, which is described in detail as follows:

[0101] S201, integrates the operational stake number and restricted flight zone layers on the high-precision map of the highway.

[0102] In this embodiment, a highway that needs to plan a route is selected, and on the three-dimensional map of the selected highway, the operating pile number is superimposed on the map based on the highway's own operating pile number and the basic data of the pile number's longitude and latitude; according to the up and down directions of the highway, the operating pile number on the up lane side of the highway is recorded as the up pile number, and the operating pile number on the down lane side of the highway is recorded as the down pile number.

[0103] Based on the three-dimensional map of the expressway, a restricted flight zone is planned on the map. If drones are prohibited from flying in the restricted flight zone, the restricted flight type is selected as a no-fly zone. If flying is allowed but only within a specified altitude range, the restricted flight type is selected as a restricted altitude zone. Among them, a first method for planning a restricted flight zone is to select multiple points on the map, connect the first point with the second point, connect the second point with the third point, and so on, until the last point is connected with the first point to form a closed area as a restricted flight zone; a second method is to select a point on the map as the center of a circle, and then select another point as a point on the circle, and generate a circle with the line between the two points as the radius as the restricted flight zone; a third method is to import longitude and latitude data on the map based on the longitude and latitude of the restricted flight zone boundary provided offline to generate a restricted flight zone.

[0104] S202, planning UAV waypoints based on the high-precision map of the highway to obtain a set of waypoints on the highway.

[0105] In this embodiment, a waypoint is selected in the high-precision map and the waypoint altitude is set. Based on the latitude, longitude and altitude of the waypoint, if the waypoint is detected to be located in the highway pavement area, or in the restricted flight zone and exceeds the restricted flight zone requirements, it is required to reselect the waypoint.

[0106] Generate the waypoint information that has been tested and is compliant, including the waypoint longitude and latitude, the waypoint altitude, the associated stake number, and the waypoint direction.

[0107] Through the function G = || W i T j ||Calculate the distance between the waypoint and the operating pile number, and take the operating pile number closest to the waypoint as the associated pile number of the waypoint; if the associated pile number of the waypoint is the uplink pile number, the waypoint is the uplink waypoint, and if the associated pile number of the waypoint is the downlink pile number, the waypoint is the downlink waypoint; the function G is the i-th waypoint, T j is the jth operation pile number, ||W i T j || is the Euclidean distance between the waypoint and the operating pile number; in the spatial rectangular coordinate system, W i The coordinates (X i ,Y i ,Z i ), T j The coordinates (X j ,Y j ,Zj ) is calculated by the function X=L*R*π / 180, Y is calculated by the function Y=ln(tan((90+K)*π / 360))*R, and Z is obtained by calculating the distance from the waypoint and the operating stake to the plane formed by the prime meridian of the earth and the east direction, where R is the radius of the earth, L represents the longitude of the waypoint or the operating stake, and L represents the latitude of the waypoint or the operating stake.

[0108] The uplink waypoints are sorted to determine the flight order between the waypoints, such as Figure 3 As shown, the specific method is to search for the associated stake numbers of the upward waypoints one by one along the upward direction, record the upward waypoint sequence number corresponding to the first associated stake number found as 1, and the upward waypoint sequence number corresponding to the second associated stake number found as 2, and so on, until all the associated stake numbers in the upward direction are searched, and the upward waypoints are sorted in ascending order according to their sequence numbers, and the UAV flies from the waypoint with a smaller sequence number to the waypoint with a larger sequence number.

[0109] The downlink waypoints are sorted to determine the flight order between the waypoints, such as Figure 4 As shown, the specific method is to search for the associated pile numbers of the downstream waypoints one by one along the downstream direction, record the downstream waypoint sequence number corresponding to the first associated pile number found as 1, and the downstream waypoint sequence number corresponding to the second associated pile number found as 2, and so on, until all the associated pile numbers in the downstream direction are searched, and the downstream waypoints are sorted in ascending order according to their sequence numbers, and the UAV flies from the waypoint with a smaller sequence number to the waypoint with a larger sequence number.

[0110] Two consecutive waypoints are connected by a straight line to form a route segment. The route segment between the two consecutive waypoints is subjected to safety and collision detection. If it is detected that the longitude and latitude of the route segment is within the restricted flight area and exceeds the requirements of the restricted flight area, or the route segment collides with the high-precision map model of the highway, the corresponding route segment is cancelled and the waypoint is required to be adjusted.

[0111] Store the planned waypoints, the flight sequence between waypoints, and the route segment information between waypoints to form a waypoint set.

[0112] S203, based on the starting point and the ending point of the UAV route, automatically obtain the first waypoint and the last waypoint of the route from the waypoint set.

[0113] In this embodiment, the drone model is selected according to the drone mission requirements, and the starting point and end point of the route are selected from the high-precision map. If it is detected that the starting point and end point are within the highway road surface area, or within the restricted flight zone and exceed the restricted flight zone requirements, it is required to reselect the starting point and end point.

[0114] A circle is generated on the horizontal plane with the selected starting point as the center and the longest flight distance of the selected model of drone as the radius. A waypoint that is within the circle and closest to the center is automatically matched from the set of waypoints on the current highway as the first waypoint of the route. If no match is found, the starting point is required to be reselected.

[0115] A circle is generated on the horizontal plane with the selected end point as the center and the longest flight distance of the selected model of drone as the radius. A waypoint that is within the circle and closest to the center is automatically matched from the waypoint set of the current highway as the end point of the route. If no match is found, the end point is required to be reselected.

[0116] Check whether the first waypoint and the last waypoint are in the same waypoint set. If yes, they are determined as the first waypoint and the last waypoint of the route, and the waypoint set where the first waypoint and the last waypoint are located is determined as the target set. Otherwise, it is required to readjust the starting point or the end point.

[0117] S204, automatically matching the best waypoint from the waypoint set where the first waypoint and the last waypoint are located to obtain the target waypoint.

[0118] In this embodiment, if the first waypoint and the last waypoint are both uplink waypoints or both downlink waypoints, such as Figure 5 As shown, all waypoints in the target set whose sequence numbers are between the first waypoint and the last waypoint are taken as the best waypoints, and the first waypoint, the last waypoint and the best waypoint are taken as the target waypoints of the planned route.

[0119] If the first waypoint is the up waypoint, the last waypoint is the down waypoint, such as Figure 6 As shown, the upward waypoint whose waypoint sequence number in the target set is greater than the sequence number of the first waypoint is taken as the best waypoint, and the downward waypoint whose waypoint sequence number in the target set is less than the sequence number of the last waypoint is taken as the best waypoint; and the first waypoint, the last waypoint, and the best waypoint are taken as the target waypoints of the planned route.

[0120] If the first waypoint is a downlink waypoint, the last waypoint is an uplink waypoint, such as Figure 7 As shown, the downlink waypoint whose waypoint sequence number in the target set is greater than the sequence number of the first waypoint is taken as the best waypoint, and the uplink waypoint whose waypoint sequence number in the target set is less than the sequence number of the last waypoint is taken as the best waypoint; and the first waypoint, the last waypoint and the best waypoint are taken as the target waypoints of the planned route.

[0121] S205, establishing a flight sequence between the target waypoints to obtain a UAV route.

[0122] In this embodiment, the flight sequence between the target waypoints has been determined when planning the waypoint set, is directly obtained, and the route is generated based on the flight sequence between the target waypoints.

[0123] Through the function Calculate the total length of the route, where is the i-th target waypoint of the drone in the three-dimensional space route, is the Euclidean distance between two adjacent waypoints in the route, H1 is the altitude of the first waypoint, H n The height of the final waypoint. If the total length of the route exceeds the maximum flight distance of the drone model, it is required to adjust the starting point or end point.

[0124] S206, planning personalized parameters of the target waypoints in the UAV route to generate a final route.

[0125] In this embodiment, based on the requirements of the drone application scenarios, the target waypoint information and route segment information are adjusted in a personalized manner, including adjusting the drone's flight speed, flight speed, gimbal pitch angle, and gimbal zoom ratio at the target waypoint, adding waypoint actions to take pictures, start recording, end recording, and hover and wait, adjusting the drone's altitude and flight speed when it is at the route segment position, adding out-of-control actions, route end actions, and yaw angle modes.

[0126] Based on the requirements of the drone application scenarios, delete the target waypoints, move the target waypoints, add new target waypoints, reselect the starting point, and reselect the end point. According to the personalized route, safety detection and collision detection are also performed. The detected result is stored as the final drone route, otherwise the route is required to be adjusted.

[0127] Reference Figure 8 An embodiment of the present invention provides an intelligent route planning system 8 for a highway drone, and the system 8 specifically includes:

[0128] The first planning module 801 is used to perform three-dimensional scanning of the highway using laser radar scanning technology to obtain three-dimensional point cloud data, segment the three-dimensional point cloud data using a point cloud segmentation algorithm, determine various structures of the highway, and construct a three-dimensional terrain model of the highway according to the type and geometric characteristics of each structure;

[0129] The second planning module 802 is used to obtain the restricted flight area data of the expressway, construct a three-dimensional restricted flight model according to the restricted flight area data, and generate a target three-dimensional model according to the three-dimensional terrain model and the three-dimensional restricted flight model;

[0130] A third planning module 803 is used to plan UAV waypoints on the highway based on the target three-dimensional model to obtain a waypoint set of the highway;

[0131] The fourth planning module 804 is used to obtain the historical route data of the highway, analyze the historical route data using a deep learning algorithm, extract the safe flight path features, and construct a route safety assessment model, wherein the route safety assessment model is used to assess the route safety level of the UAV route;

[0132] The fifth planning module 805 is used to determine the route planning parameters according to the UAV mission type, and determine the UAV route through the waypoint set, the route safety assessment model and the waypoint planning parameters based on a multi-objective optimization algorithm;

[0133] The sixth planning module 806 is used to use a convolutional neural network to perform real-time analysis on the aerial images, identify and determine the location and movement trend of obstacles, and dynamically adjust the UAV route according to the location and movement trend of the obstacles.

[0134] It is understandable that if Figure 1 The contents of the embodiment of the intelligent route planning method for a highway drone shown in the figure are applicable to the embodiment of the intelligent route planning system for a highway drone. The functions specifically implemented by the embodiment of the intelligent route planning system for a highway drone are similar to those in the embodiment of the figure. Figure 1 The embodiment of the intelligent route planning method for a highway drone shown in FIG. 1 is the same as that of the embodiment of the intelligent route planning method for a highway drone shown in FIG. Figure 1 The beneficial effects achieved by the embodiment of the intelligent route planning method for highway drones shown are also the same.

[0135] It should be noted that the information interaction, execution process and other contents between the above-mentioned systems are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0136] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0137] Reference Fig. 9 The embodiment of the present invention further provides a computer device 9, comprising: a memory 902 and a processor 901 and a computer program 903 stored in the memory 902. When the computer program 903 is executed on the processor 901, the intelligent route planning method for a highway drone as described in any one of the above methods is implemented.

[0138] The computer device 9 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 9 may include, but is not limited to, a processor 901 and a memory 902. Those skilled in the art will appreciate that Fig. 9 It is only an example of the computer device 9 and does not constitute a limitation on the computer device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0139] The processor 901 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0140] In some embodiments, the memory 902 may be an internal storage unit of the computer device 9, such as a hard disk or memory of the computer device 9. In other embodiments, the memory 902 may also be an external storage device of the computer device 9, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 9. Further, the memory 902 may also include both an internal storage unit of the computer device 9 and an external storage device. The memory 902 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as the program code of the computer program, etc. The memory 902 may also be used to temporarily store data that has been output or is to be output.

[0141] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for intelligent route planning for a highway drone as described in any one of the above methods is implemented.

[0142] In this embodiment, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0143] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0144] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0145] In the embodiments disclosed in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0146] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Claims

1. An intelligent route planning method for a highway drone, characterized in that: The method specifically comprises: Using laser radar scanning technology to perform three-dimensional scanning of the highway to obtain three-dimensional point cloud data, segmenting the three-dimensional point cloud data using a point cloud segmentation algorithm to determine various structures of the highway, and constructing a three-dimensional terrain model of the highway according to the type and geometric characteristics of each structure; Acquire flight restriction area data of the expressway, construct a three-dimensional flight restriction model based on the flight restriction area data, and generate a target three-dimensional model according to the three-dimensional terrain model and the three-dimensional flight restriction model; Planning UAV waypoints on the highway based on the target three-dimensional model to obtain a waypoint set of the highway; Acquire historical route data of the expressway, analyze the historical route data using a deep learning algorithm, extract safe flight path features, and construct a route safety assessment model, wherein the route safety assessment model is used to assess the route safety level of the UAV route; Determine route planning parameters according to the type of UAV mission, and determine the UAV route through the waypoint set, the route safety assessment model and the waypoint planning parameters based on a multi-objective optimization algorithm; A convolutional neural network is used to perform real-time analysis of aerial images to identify and determine the location and movement trend of obstacles, and the UAV route is dynamically adjusted according to the location and movement trend of the obstacles.

2. The method according to claim 1, characterized in that: The types of structures include bridge structures, tunnel structures and slope structures; and constructing the three-dimensional terrain model of the highway according to the types and geometric features of each structure specifically includes: The three-dimensional point cloud data of the bridge structure is converted into a triangular mesh surface by using a triangular mesh method, and the triangular mesh surface is processed by a surface reconstruction technology to obtain a first three-dimensional terrain model of the bridge structure; Using a cylindrical surface fitting method to fit the three-dimensional point cloud data of the tunnel structure into a plurality of cylindrical surfaces, and by splicing the plurality of cylindrical surfaces, a second three-dimensional terrain model of the tunnel structure is obtained; The three-dimensional point cloud data of the slope structure is converted into a polygonal mesh surface by using a polygonal meshing method, and the polygonal mesh surface is processed by a surface reconstruction technology to obtain a third three-dimensional terrain model of the slope structure.

3. The method according to claim 1, characterized in that: The step of constructing a three-dimensional flight restriction model using the flight restriction area data and generating a target three-dimensional model according to the three-dimensional terrain model and the three-dimensional flight restriction model specifically includes: Determine the vertical range, horizontal range and flight restriction requirements at different altitudes of the restricted flight area according to the restricted flight area data; Determine the road section range affected by the restricted flight area of ​​the expressway according to the vertical range and the horizontal range; Determine the flight altitude limit for each location within the route section according to the flight restriction requirements at different altitudes; Constructing a three-dimensional flight restriction model according to the road section range and the flight altitude restrictions of various locations within the road section range; After the spatial coordinate systems of the three-dimensional terrain model and the three-dimensional flight restriction model are unified and registered, the three-dimensional terrain model and the three-dimensional flight restriction model are subjected to a three-dimensional model merging operation through a three-dimensional grid stitching algorithm to generate a target three-dimensional model.

4. The method according to claim 1, characterized in that: The step of planning a UAV waypoint on the highway based on the target three-dimensional model to obtain a waypoint set of the highway specifically includes: Based on the target three-dimensional model, the expressway is spatially divided using a three-dimensional space division algorithm to obtain a spatial grid model of the expressway; According to the highway spatial grid model, a three-dimensional spatial path planning algorithm is used to plan a flight path of the UAV on the highway to obtain a waypoint flight path of the UAV; A three-dimensional space sampling algorithm is used to perform waypoint sampling on the waypoint flight path to obtain a sampled waypoint set.

5. The method according to claim 1, characterized in that The use of a deep learning algorithm to analyze the historical route data, extract safe flight path features, and construct a route safety assessment model specifically includes: Using a long short-term memory neural network to extract features from the historical route data to obtain safe flight path features; According to the safe flight path characteristics, a first route safety assessment model is obtained by performing classification training through a support vector machine, where the first route safety assessment model is used to determine whether the route is safe; Acquiring safety assessment index data according to the historical route data, the safety assessment index data including collision risk data, meteorological condition data and electromagnetic interference data; Use fuzzy comprehensive evaluation method to quantify and score each safety assessment index data to obtain the evaluation score of each safety assessment index data; The weighted average method is used to sum the evaluation scores of each safety evaluation index data to obtain the comprehensive safety score of the route; According to the comprehensive safety score of the route, the route is divided into safety levels by using a cluster analysis method to obtain a set of routes with different safety levels; Taking the route set of different safety levels and each safety assessment index data as input, a second route safety assessment model is constructed through a decision tree algorithm, and the second route safety assessment model is used to assess the safety level of the route.

6. The method according to claim 1, characterized in that The determining of the route planning parameters according to the UAV mission type, based on the multi-objective optimization algorithm, determines the UAV route through the waypoint set, the route safety assessment model and the waypoint planning parameters, specifically includes: By constructing a mapping relationship between the UAV mission type and the route planning parameters, the route planning parameters under different mission types are obtained, wherein the route planning parameters include flight altitude, speed and heading angle; According to the importance of the UAV mission type, the weight of each route planning parameter is calculated using the hierarchical analysis method to obtain the weight vector of route planning constraints; Taking route length, flight time and flight energy consumption as optimization targets, taking the waypoint set, the evaluation result of the route safety evaluation model and the route planning constraint weight vector as constraint conditions, adopting a multi-objective optimization algorithm to solve the route planning, and obtaining a Pareto optimal solution set; The optimal route that meets the mission requirements is selected from the Pareto optimal solution set as the planning result to determine the UAV route.

7. The method according to claim 1, characterized in that The dynamically adjusting the route of the UAV according to the position and movement trend of the obstacle specifically includes: Acquire the current drone position data, fuse the position and movement trend of the obstacle with the current drone position data, use the Kalman algorithm to recursively estimate the drone's position and speed, and obtain the drone position estimation result; The drone position estimation result is used as a constraint condition to optimize the drone route in real time.

8. An intelligent route planning system for highway drones, characterized in that: The system specifically comprises: The first planning module is used to perform three-dimensional scanning of the highway using laser radar scanning technology to obtain three-dimensional point cloud data, segment the three-dimensional point cloud data using a point cloud segmentation algorithm, determine various structures of the highway, and construct a three-dimensional terrain model of the highway according to the type and geometric characteristics of each structure; A second planning module is used to obtain flight restriction area data of the expressway, construct a three-dimensional flight restriction model based on the flight restriction area data, and generate a target three-dimensional model according to the three-dimensional terrain model and the three-dimensional flight restriction model; A third planning module is used to plan UAV waypoints on the highway based on the target three-dimensional model to obtain a waypoint set of the highway; The fourth planning module is used to obtain the historical route data of the expressway, analyze the historical route data using a deep learning algorithm, extract the safe flight path features, and construct a route safety assessment model, wherein the route safety assessment model is used to assess the route safety level of the UAV route; A fifth planning module, used to determine the route planning parameters according to the UAV mission type, and determine the UAV route based on the waypoint set, the route safety assessment model and the waypoint planning parameters based on a multi-objective optimization algorithm; The sixth planning module is used to use a convolutional neural network to perform real-time analysis of aerial images, identify and determine the location and movement trend of obstacles, and dynamically adjust the UAV route according to the location and movement trend of the obstacles.

Images