Unmanned aerial vehicle route planning system and method based on visual image

Abstract

The invention belongs to the technical field of unmanned aerial vehicle route planning, and particularly relates to an unmanned aerial vehicle route planning system and method based on a visual image. According to the method, the route of the unmanned aerial vehicle is precisely planned, the problem that route planning is inaccurate due to complex terrain and numerous obstacles in a traditional method is effectively avoided, deviation points in obstacle avoidance factors can be found and corrected in time by collecting and analyzing visual image information in real time, and the obstacle avoidance accuracy is improved. Therefore, the safety and stability of the unmanned aerial vehicle in the flight process are ensured, meanwhile, the invention further provides a scheme for optimizing an ideal path based on the corrected obstacle avoidance factors, the feasibility and efficiency of route planning are further improved, and in addition, through comprehensive evaluation of various flight parameters, the route planning efficiency is improved. According to the method, scientific priority ranking can be carried out on each executable path, so that intelligent selection of the route of the unmanned aerial vehicle is realized, the flight efficiency of the unmanned aerial vehicle is improved, and the energy consumption in the flight process is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle (UAV) route planning, and in particular relates to a UAV route planning system and method based on visual images. Background Art

[0002] At present, when planning UAV routes, the impact of obstacles on flight is generally given priority. Therefore, when planning routes, obstacle avoidance is generally optimized. The conventional means of dealing with obstacles is to choose to bypass them so as to reach the operation point in time. However, in the actual flight process, the higher the obstacle avoidance accuracy, the greater the impact on the UAV's battery life. Therefore, when planning UAV routes, it is necessary to combine the actual path with the UAV's battery life and plan the flight path in advance to reduce the high energy consumption caused by obstacle identification and judgment during subsequent flights. This is also the current mainstream planning method.

[0003] Although the above method can greatly improve the route planning ability of drones, the choice of drones in planning routes is generally determined by flight distance, that is, when avoiding obstacles, paths with shorter distances are given priority, while the actual operating status of drones under each path, such as drone flight attitude, sensor operating frequency, etc., are ignored. This not only has a great impact on the comprehensive endurance of the drone, but also dense obstacles may appear in shorter paths, which will undoubtedly increase the difficulty of drone flight missions. When obstacles are dense, the drone's attitude changes will increase and its speed will decrease, which will shorten the actual endurance of the drone and affect actual operations. Summary of the invention

[0004] The purpose of the present invention is to provide a UAV route planning system and method based on visual images, which can comprehensively consider various factors of the UAV during the flight process, such as density, altitude difference, flight time, path length, number of flight attitude adjustments, obstacle avoidance difficulty, etc., so as to plan a more reasonable and efficient flight path.

[0005] The technical solution adopted by the present invention is as follows: A method for UAV route planning based on visual images, comprising: Obtain map data of the road section to be planned, plan an initial path in the map data in the form of the shortest path, optimize the initial path based on influencing factors to obtain multiple ideal paths, and mark obstacle avoidance factors in each ideal path, which include obstacles and terrain; Select multiple ideal paths as the planned routes of the drone, and turn on the automatic obstacle avoidance function of the drone to the maximum power consumption, so that the drone can obtain visual image information in the ideal path during flight, and use the obstacle images and terrain images captured during flight to determine the deviation points of obstacle avoidance factors based on the visual images, and correct the deviation points in the ideal path; Based on the corrected obstacle avoidance factors, the ideal path is optimized, multiple executable paths are planned, and the flight parameters of each executable path are collected. The executable paths are prioritized according to the flight parameters, where the flight parameters include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor energy consumption; Based on the priority ranking, the drone route is selected, and the priority ranking is adjusted according to the visual image information of the actual flight process.

[0006] In a preferred embodiment, the step of optimizing the initial path based on the influencing factors to obtain multiple ideal paths includes: Get the no-fly zone geofence data and mark it in the map data; The portion of the initial path that crosses the no-fly zone is offset to both sides to obtain multiple paths to be processed, wherein when the initial path includes multiple no-fly zones and the distance between adjacent no-fly zones is less than a preset distance, the adjacent no-fly zones are merged and regarded as one no-fly zone; Filter multiple pending paths. Based on the drone's cruising range, remove the pending paths whose path length is greater than the cruising range, and remove the pending paths that still pass through the no-fly zone after being offset. The multiple paths to be processed after screening are respectively recorded as ideal paths.

[0007] In a preferred embodiment, the step of determining the deviation point of the obstacle avoidance factor in the visual image and correcting the deviation point in the ideal path includes: Obtain obstacle images and terrain images in the visual image, process the obstacle images and terrain images, and obtain actual parameters of the obstacles and terrain; Compare the actual parameters of obstacles and terrain with the obstacles and terrain in the obstacle avoidance factors to determine the deviation points, where the deviation points include position and specification deviations, missing or newly added deviations; According to the deviation points, the obstacle avoidance factors are corrected on the ideal path.

[0008] In a preferred embodiment, after the obstacle avoidance factor on the ideal path is corrected, the correction result can be automatically memorized and applied when the route planning task is executed next time, and the frequency of visual image acquisition of the drone is increased before reaching the obstacle avoidance factor. In a preferred embodiment, the step of optimizing the ideal path based on the corrected obstacle avoidance factors and planning multiple executable paths includes: Based on the optimized ideal path, combined with the outline, size and location information of obstacles and terrain in the visual image information, the feasibility analysis of each ideal path is carried out, and the ideal path is classified into executable path and non-executable path; The non-executable paths are removed from the map, and only the executable paths are retained.

[0009] In a preferred solution, the step of performing feasibility analysis on each ideal path and classifying the ideal path into executable paths and non-executable paths includes: Extracting density, height difference and flight time information of obstacles and terrain from each of the ideal paths; Evaluate the risk factor of each ideal path based on the density, altitude difference and flight time information; comparing the risk factor with a preset classification threshold; When the risk coefficient is lower than the classification threshold, it indicates that the ideal path corresponding to the risk coefficient has high safety, and the route corresponding to the ideal path is recorded as an executable path; When the risk coefficient is higher than or equal to the classification threshold, it indicates that the safety of the ideal path corresponding to the risk coefficient is low, and the route corresponding to the ideal path is recorded as a non-executable path.

[0010] In a preferred embodiment, the step of selecting the UAV route based on the priority sorting and adjusting the priority sorting according to the visual image information of the actual flight process includes: The default preferred flight path is the drone route; Collect obstacle and terrain images captured by the drone in real time under the preferred flight path, and compare the obstacle and terrain images captured in real time with the planned preferred flight path; When the obstacle and terrain images captured in real time are not in the preferred flight path, a detour path is generated, and the degree of fit between the detour path and the preferred flight path and the alternative flight path is matched; The drone route is reselected based on the degree of adaptability. If the detour path has the highest degree of adaptability with the original preferred flight path, the flight continues according to the original detour path. Otherwise, the alternative flight path with the highest degree of adaptability is selected for flight, and the drone route planning is updated at the same time.

[0011] The present invention also provides a UAV route planning system based on visual images using the above-mentioned UAV route planning method based on visual images, comprising: Obtain map data of the road section to be planned, plan an initial path in the map data in the form of the shortest path, optimize the initial path based on influencing factors to obtain multiple ideal paths, and mark obstacle avoidance factors in each ideal path, which include obstacles and terrain; Select multiple ideal paths as the planned routes of the drone, and turn on the automatic obstacle avoidance function of the drone to the maximum power consumption, so that the drone can obtain visual image information in the ideal path during flight, and use the obstacle images and terrain images captured during flight to determine the deviation points of obstacle avoidance factors based on the visual images, and correct the deviation points in the ideal path; Based on the corrected obstacle avoidance factors, the ideal path is optimized, multiple executable paths are planned, and the flight parameters of each executable path are collected. The executable paths are prioritized according to the flight parameters, where the flight parameters include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor energy consumption; Based on the priority ranking, the drone route is selected, and the priority ranking is adjusted according to the visual image information of the actual flight process.

[0012] And, an electronic device, the electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned visual image-based drone route planning method.

[0013] The technical effects achieved by the present invention are: The present invention effectively avoids the problem of inaccurate route planning caused by complex terrain and numerous obstacles in traditional methods by accurately planning the UAV route. By real-time collection and analysis of visual image information, the present invention can timely discover and correct deviation points in obstacle avoidance factors, thereby ensuring the safety and stability of the UAV during flight. At the same time, the present invention also provides a solution for optimizing the ideal path based on the corrected obstacle avoidance factors, further improving the feasibility and efficiency of route planning. In addition, through a comprehensive evaluation of multiple flight parameters, the present invention can scientifically prioritize each executable path, thereby realizing intelligent selection of UAV routes, which not only improves the flight efficiency of the UAV, but also reduces energy consumption during flight, providing strong technical support for the widespread application of UAVs.

[0014] The present invention plans the route of the UAV so that when the UAV plans the flight route, the sensor power can be adjusted according to the positions of obstacles and terrain in the route. When the obstacles are dense, the obstacle avoidance capability is enhanced, that is, the sensor power is increased, and the power in other areas is reduced, that is, the reasonable use of each sensor is guaranteed, especially for the deviation point part. When passing through the deviation point area, the obstacle avoidance capability is also appropriately increased to avoid temporary changes, thereby enhancing the safe flight capability of the UAV. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic flow chart of the method of the present invention; Figure 2 It is a schematic diagram of the system module of the present invention; Figure 3 It is a schematic diagram of the structure of an electronic device of the present invention. DETAILED DESCRIPTION

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0017] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0018] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure or characteristic that may be included in at least one implementation of the present invention. The term "in a preferred embodiment" that appears in different places in this specification does not refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0019] See also Figure 1 As shown, the present invention provides a UAV route planning method based on visual images, comprising: S1. Obtain map data of the road section to be planned, plan an initial path in the map data in the form of the shortest path, optimize the initial path based on influencing factors to obtain multiple ideal paths, and mark obstacle avoidance factors in each ideal path, where the obstacle avoidance factors include obstacles and terrain; In step S1, when planning the route of the drone, it is first necessary to obtain map data of the road section to be planned, and then plan an initial path in the map data in the form of the shortest path. This initial path corresponds to the shortest physical distance and does not consider factors such as obstacles, terrain, and no-fly zones. Then, the initial path is optimized based on various influencing factors to obtain multiple ideal paths. In each ideal path, obstacle avoidance factors need to be marked. The obstacle avoidance factors mainly include obstacles and terrain, etc., wherein the steps of optimizing the initial path based on the influencing factors to obtain multiple ideal paths include: Get the no-fly zone geofence data and mark it in the map data; The portion of the initial path that crosses the no-fly zone is offset to both sides to obtain multiple paths to be processed, wherein when the initial path includes multiple no-fly zones and the distance between adjacent no-fly zones is less than a preset distance, the adjacent no-fly zones are merged and regarded as one no-fly zone; Filter multiple pending paths. Based on the drone's cruising range, remove the pending paths whose path length is greater than the cruising range, and remove the pending paths that still pass through the no-fly zone after being offset. The multiple paths to be processed after screening are respectively recorded as ideal paths; Specifically, when outputting the ideal path, it is first necessary to obtain the geographic fence data of the no-fly zone and accurately mark it on the map data to ensure that these areas can be accurately identified and avoided when planning the path, and then analyze the initial path. If it is found that the path crosses the no-fly zone, it needs to be further adjusted. The specific operation is to offset the part of the initial path that passes through the no-fly zone to both sides. In this way, multiple paths to be processed can be obtained. When offsetting, if the initial path contains multiple no-fly zones, and the distance between adjacent no-fly zones is less than a preset specific distance, then in the process of offsetting the initial path to both sides, it is necessary to consider bypassing these adjacent no-fly zones at the same time to ensure the compliance and safety of the path. For example, the distance between adjacent no-fly zones is 500m, and the preset specific distance is 100. 0m, then when offsetting the path, it is necessary to bypass the two no-fly zones at the same time (that is, the two no-fly zones are regarded as one no-fly zone. Of course, when more no-fly zones are adjacent to each other, multiple adjacent no-fly zones are regarded as one no-fly zone) to ensure that the drone will not enter the no-fly zone during flight. After obtaining multiple paths to be processed, they need to be further screened. The basis for screening is the cruising range of the drone. Specifically, those paths to be processed whose path length exceeds the cruising range of the drone will be eliminated, and their corresponding paths are unrealistic. At the same time, those paths that will still pass through other no-fly zones even after offset processing will also be eliminated. After this screening work, the remaining paths to be processed can be identified as ideal paths. The ideal path not only avoids the no-fly zone, but also is a practical and feasible flight route within the range of the drone's cruising range.

[0020] S2. Select multiple ideal paths as the planned routes of the UAV, and turn on the automatic obstacle avoidance function of the UAV to the maximum power consumption, so that the UAV can obtain visual image information in the ideal path during flight, and use the obstacle images and terrain images captured during flight to determine the deviation points of obstacle avoidance factors based on the visual images, and correct the deviation points in the ideal path; In step S2, after the ideal path is output, the automatic obstacle avoidance function of the drone is turned on to the maximum power consumption, so that during the flight of the drone, the visual image information in the ideal path can be obtained in real time, including obstacle images and terrain images captured during the flight. According to the real-time visual image information, the deviation points of the obstacle avoidance factors can be determined, and these deviation points can be corrected in the ideal path. At the same time, the automatic obstacle avoidance function of the drone is turned on to the maximum power, which can ensure that the drone has efficient obstacle avoidance capabilities when testing the path. In this state, each sensor (such as radar, etc.) can accurately sense obstacles and terrain, so that when the drone is tested, the environment in the path, in combination with the visual image information, can more easily mark the obstacles and terrain in the path. Among them, the steps of determining the deviation points of the obstacle avoidance factors in the visual image and correcting the deviation points in the ideal path include: Obtain obstacle images and terrain images in the visual image, process the obstacle images and terrain images, and obtain actual parameters of the obstacles and terrain; Compare the actual parameters of obstacles and terrain with the obstacles and terrain in the obstacle avoidance factors to determine the deviation points, where the deviation points include position and specification deviations, missing or newly added deviations; According to the deviation points, the obstacle avoidance factors are corrected on the ideal path; Specifically, when an obstacle avoidance factor that is not marked in the ideal path appears in the real-time visual image information, the obstacle position coordinates acquired in real time are first collected. , and then compare it with the coordinates of the obstacle avoidance factors preset in the ideal path Compare and output the initial distance deviation, where the calculation formula for the initial distance deviation is: , where Represents the initial distance deviation, where the position and specification deviation, confirmed or newly added deviation can be determined based on the initial distance deviation. The position and specification deviation is the deviation between the actual position of the obstacle avoidance factor in the real-time image and the preset position in the ideal path. The missing deviation refers to the obstacle avoidance factor preset in the ideal path not appearing in the real-time image. The newly added deviation refers to the obstacle avoidance factor not preset in the ideal path appearing in the real-time image. In this way, the actual impact of the obstacle avoidance factor on the UAV route can be determined more accurately, so as to make corresponding corrections on the ideal path. The correction process includes the following two aspects: For position and specification deviations, a local path replanning method is used to generate a detour trajectory based on the original ideal path and meet the minimum turning radius constraint of the drone: ; In the formula, Indicates the minimum turning radius, Indicates the flight speed of the drone. represents the acceleration due to gravity, Indicates the maximum roll angle; For newly added obstacles, temporary waypoints need to be inserted into the path, and the new and old waypoints need to be smoothly connected using cubic spline interpolation to ensure that the drone can smoothly transition during flight and avoid flight instability or safety risks caused by sudden changes in flight direction; In addition, missing deviations, that is, preset obstacle avoidance factors that do not appear in real-time images, will be reconfirmed to ensure whether they have indeed disappeared due to changes in environmental factors. If confirmed to have disappeared, they will be deleted from the ideal path to avoid unnecessary flight adjustments. In this way, the drone can accurately adjust the flight route based on real-time visual image information, thereby effectively avoiding obstacles and terrain obstacles and ensuring flight safety and efficiency.

[0021] Finally, the map data needs to be updated and the confidence of the obstacle position is updated by probabilistically occupying the grid: ; In the formula, Indicates at time All observation data in Next, the entire environment map The probability distribution of Represents the first units (each unit represents a spatial region), Represents the probability that a single grid cell is occupied by an obstacle; In addition, after the obstacle avoidance factors on the ideal path are corrected, the next time the route planning task is executed, the correction results can be automatically memorized and applied, and the frequency of UAV visual image acquisition will be increased before reaching the obstacle avoidance factor to ensure more accurate identification and response to potential obstacles. If the corrected obstacle avoidance factors do not appear in the route later, the frequency of UAV visual image acquisition will be gradually reduced to save energy and improve flight efficiency. If the corrected obstacle avoidance factors appear multiple times, they will be recorded as regular obstacles and added to the map data. In the subsequent route execution, such regular obstacles will be treated as fixed obstacle avoidance factors, and there is no need for real-time image acquisition and correction, so as to further optimize the UAV route planning.

[0022] S3. Based on the corrected obstacle avoidance factors, the ideal path is optimized, multiple executable paths are planned, and flight parameters of each executable path are collected, and each executable path is prioritized according to the flight parameters, wherein the flight parameters include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor energy consumption; In step S3, based on the corrected obstacle avoidance factors, the ideal path is further optimized to plan multiple executable paths. Subsequently, the flight parameters of each executable path are collected, including path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor energy consumption. The executable paths are prioritized according to the flight parameters. The steps of optimizing the ideal path based on the corrected obstacle avoidance factors and planning multiple executable paths include: Based on the optimized ideal path, combined with the outline, size and location information of obstacles and terrain in the visual image information, the feasibility analysis of each ideal path is carried out, and the ideal path is classified into executable path and non-executable path; Remove non-executable paths from the map and keep only executable paths; Specifically, after considering the corrected obstacle avoidance factors, the ideal path needs to be optimized accordingly. First, based on the optimized ideal path, the outline, size and location information of the obstacles and terrain are further analyzed in combination with the visual image information. Based on this, the feasibility analysis can be performed on each ideal path, so that the ideal paths are divided into two categories: one is the path with the possibility of execution, and the other is the path without the execution conditions. Finally, those paths that are judged as non-executable will be removed from the map to ensure that only verified and feasible execution paths are retained on the map to facilitate subsequent navigation and path planning. Secondly, the feasibility analysis of each ideal path is performed, and the steps of classifying the ideal path into executable paths and non-executable paths include: Extract obstacle and terrain density, height difference, and flight time information from each ideal path; Evaluate the risk factor of each ideal path based on density, altitude difference and flight time information; Compare the risk factor with a preset classification threshold; When the risk factor is lower than the classification threshold, it indicates that the ideal path corresponding to the risk factor has high safety, and the route corresponding to the ideal path is recorded as an executable path; When the risk factor is higher than or equal to the classification threshold, it indicates that the safety of the ideal path corresponding to the risk factor is low, and the route corresponding to this ideal path is recorded as a non-executable path; In this implementation, when dividing executable paths and non-executable paths, first, key information such as density of obstacles and terrain, height difference, and flight time is extracted from each ideal path; Wherein, when determining the density of obstacles and terrain, it is determined based on the obstacle density calculation function; ; In the formula, Indicates the density of obstacles and terrain, Represents the total number of obstacles in the path area, represents the path length, Indicates the safe flight width of the drone (no less than twice the width of the fuselage); The height difference evaluation function is: ; In the formula, Indicates the height difference, Represents the path sampling point Altitude above sea level; The flight time estimation function is: ; In the formula, Indicates the estimated flight time. represents the average flight speed of the drone, Indicates the number of times the UAV's flight attitude is adjusted. Indicates the time taken for the drone to adjust its flight attitude in a single flight; After the obstacle density, flight height difference and estimated flight time in the ideal path are output, they can be input into the preset comprehensive evaluation function to evaluate the risk factor of each ideal path. The expression of the comprehensive evaluation function is: ; In the formula, represents the risk factor, , and They represent the preset maximum density, altitude difference and flight time (set according to mission requirements). , and They represent the weight coefficients of obstacle density, height difference and flight time respectively. Then, the calculated risk coefficient is compared with the preset classification threshold. If it is found that the risk coefficient of an ideal path is lower than the set classification threshold, then it can be considered that the ideal path has higher safety. Therefore, the route corresponding to this ideal path is recorded as an executable path. On the contrary, if the risk coefficient is higher than or equal to the classification threshold, it means that the safety of the ideal path is lower. Therefore, the route corresponding to this ideal path is recorded as a non-executable path. It should be further explained that when the executable paths are prioritized according to the flight parameters, the risk coefficients under each executable path are arranged in descending order. , where the executable path with a smaller risk coefficient has a higher priority, and the executable path with a larger risk coefficient has a lower priority. When there are multiple executable paths with the same risk coefficient and the highest ranking, the path lengths are further compared, and the executable path with the shortest path length is determined as the highest priority path. If the path lengths are also the same, the number of flight attitude adjustments is compared, and the executable path with the least number of flight attitude adjustments is determined as the highest priority path. If the number of flight attitude adjustments is also the same, the sensor energy consumption is compared, and the executable path with the lowest sensor energy consumption is determined as the highest priority path. In this way, it can be ensured that the drone selects the safest and most efficient flight path when performing a mission.

[0023] S4, based on the priority ranking, select the drone route, and adjust the priority ranking according to the visual image information of the actual flight process; In step S4, after the priority ranking of the executable paths is determined, the UAV route is selected based on the priority ranking, and during the actual flight process, the priority ranking is dynamically adjusted according to the visual image information obtained in real time to ensure that the UAV can safely complete the flight mission, wherein the steps of selecting the UAV route based on the priority ranking and adjusting the priority ranking according to the visual image information of the actual flight process include: The default preferred flight path is the drone route; Collect obstacle and terrain images captured by the drone in real time under the preferred flight path, and compare the obstacle and terrain images captured in real time with the planned preferred flight path; When the obstacle and terrain images captured in real time are not in the preferred flight path, a detour path is generated, and the degree of fit between the detour path and the preferred flight path and the alternative flight path is matched; Reselect the drone route based on the adaptability. If the detour path has the highest adaptability to the original preferred flight path, continue to fly according to the original detour path. Otherwise, select the alternative flight path with the highest adaptability and update the drone route planning at the same time. Specifically, based on the priority sorting mechanism, during the actual flight process, the established priority sorting will be dynamically adjusted according to the visual image information obtained. First, the default preferred flight path will be used as the basic route for the UAV to perform the mission. Secondly, the image information of various obstacles and terrain environments captured by the UAV during the flight along the preferred flight path will be collected in real time and continuously, and the real-time captured obstacle and terrain images will be compared and analyzed in detail with the pre-planned preferred flight path. When it is found that the real-time captured obstacle and terrain images are not included in the preset range of the original preferred flight path, the emergency plan will be immediately activated to generate a corresponding detour path. There are also multiple detour paths, and the detour end position of each detour path coincides with the original preferred flight path and the alternative flight path respectively. By evaluating the compatibility between the detour path and the original preferred flight path and the alternative flight path, flight safety and mission efficiency are ensured. First, the risk coefficient of the detour path needs to be determined (this is consistent with the above-mentioned method of determining the preferred flight path by the risk coefficient, and will not be repeated here), and it is recorded as the fitness. Then, according to the fitness results obtained by the evaluation, the UAV's route is reselected. If the generated detour path has the highest fitness with the original preferred flight path, the flight continues to be carried out according to the detour path. Otherwise, the alternative flight path with the highest fitness is selected to fly to ensure that the UAV can complete the flight mission safely and efficiently. Finally, while completing the route selection, the UAV's route planning information is synchronously updated to optimize the overall execution effect of subsequent flight missions.

[0024] See also Figure 2 , a UAV route planning system based on visual images, using the above-mentioned UAV route planning method based on visual images, comprising: Initialization module: The initialization module is used to obtain map data of the road section to be planned, plan an initial path in the map data in the form of the shortest path, optimize the initial path based on influencing factors, obtain multiple ideal paths, and mark obstacle avoidance factors in each ideal path, which include obstacles and terrain; Deviation positioning module, which is used to select multiple ideal paths as the planned routes of the UAV, and turn on the automatic obstacle avoidance function of the UAV to the maximum power consumption, so that the UAV can obtain visual image information in the ideal path during flight, and is used to capture obstacle images and terrain images during flight, determine the deviation points of obstacle avoidance factors based on the visual images, and correct the deviation points in the ideal path; The screening and sorting module is used to optimize the ideal path based on the corrected obstacle avoidance factors, plan multiple executable paths, collect flight parameters of each executable path, and prioritize each executable path according to the flight parameters, where the flight parameters include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor energy consumption; Dynamic planning module, which is used to select the UAV route based on priority sorting and adjust the priority sorting according to the visual image information of the actual flight process; In the above, the main function of the initialization module is to obtain the detailed map data of the road section to be planned. After obtaining the map data, the system will preliminarily plan an initial path in the map data in the form of the shortest path. Subsequently, the system will carefully optimize this initial path based on a variety of influencing factors to generate multiple ideal paths. In the ideal path, the system will further mark various obstacle avoidance factors. These obstacle avoidance factors include not only various obstacles, but also complex terrain conditions. The role of the deviation positioning module is to select multiple optimized ideal paths as the planned routes of the UAV. In this process, the system will turn on the automatic obstacle avoidance function of the UAV to the maximum power consumption state to ensure that the UAV can efficiently obtain the visual image information in the ideal path during flight. The visual image information is mainly used to capture the obstacle images and terrain images encountered during flight. The system will use these visual image information , accurately determine the deviation points of obstacle avoidance factors, and make timely corrections to these deviation points in the ideal path. The task of the screening and sorting module is to further optimize multiple ideal paths based on the corrected obstacle avoidance factors, so as to plan multiple executable paths. The system will also collect detailed flight parameters of each executable path. The flight parameters include the total length of the path, the number of flight attitude adjustments, the difficulty of obstacle avoidance, and the energy consumption of the sensor. According to these flight parameters, the system will prioritize each executable path to ensure that the drone can choose the optimal path for flight. The function of the dynamic planning module is to make the final selection of the drone's route based on the priority sorting results. During the actual flight of the drone, the system will dynamically adjust the priority sorting according to the real-time visual image information to ensure that the drone can flexibly respond to various emergencies and ensure the safety and efficiency of the flight.

[0025] See also Figure 3 , an electronic device, the electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; Among them, the memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the above-mentioned visual image-based drone route planning method.

[0026] The processor of the electronic device may be Central processing unit (CPU), graphics processing unit (GPU) or digital signal processor (DSP), etc. The memory is composed of external memory and internal memory. The external memory includes any one of hard disk (Hard Disk), solid state drive (SSD), flash memory (Flash Memory), and the internal memory includes any one of random access memory (RAM) and read-only memory (ROM). The electronic device may also include an arithmetic unit, input device, output device, network interface, etc. The arithmetic unit may be an arithmetic logic unit (ALU) for performing various arithmetic and logical operations. The input device may be a keyboard, a mouse, a touch screen, etc. The output device may be a display, a printer, etc. The network interface may be an Ethernet interface, a wireless LAN interface, etc. Of course, the electronic device may also include other necessary components, such as a power supply, a cooling fan, etc.

[0027] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0028] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to the conventional means in the art unless otherwise specified and limited.

Claims

1. A method for UAV route planning based on visual images, characterized in that: include: Obtain map data of the road section to be planned, plan an initial path in the map data in the form of the shortest path, optimize the initial path based on influencing factors to obtain multiple ideal paths, and mark obstacle avoidance factors in each ideal path, which include obstacles and terrain; Select multiple ideal paths as the planned routes of the drone, and turn on the automatic obstacle avoidance function of the drone to the maximum power consumption, so that the drone can obtain visual image information in the ideal path during flight, and use the obstacle images and terrain images captured during flight to determine the deviation points of obstacle avoidance factors based on the visual images, and correct the deviation points in the ideal path; Based on the corrected obstacle avoidance factors, the ideal path is optimized, multiple executable paths are planned, and the flight parameters of each executable path are collected. The executable paths are prioritized according to the flight parameters, where the flight parameters include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor energy consumption; Based on the priority ranking, the drone route is selected, and the priority ranking is adjusted according to the visual image information of the actual flight process.

2. The method for UAV route planning based on visual images according to claim 1, characterized in that: The step of optimizing the initial path based on the influencing factors to obtain multiple ideal paths includes: Get the no-fly zone geofence data and mark it in the map data; The portion of the initial path that crosses the no-fly zone is offset to both sides to obtain multiple paths to be processed, wherein when the initial path includes multiple no-fly zones and the distance between adjacent no-fly zones is less than a preset distance, the adjacent no-fly zones are merged and regarded as one no-fly zone; Filter multiple paths to be processed. Based on the UAV's cruising range, remove the paths to be processed whose length is greater than the cruising range, and remove the paths to be processed that still pass through the no-fly zone after being offset. The multiple paths to be processed after screening are respectively recorded as ideal paths.

3. The method for UAV route planning based on visual images according to claim 1, characterized in that: The step of determining the deviation point of the obstacle avoidance factor according to the visual image and correcting the deviation point in the ideal path includes: Obtain obstacle images and terrain images in the visual image, process the obstacle images and terrain images, and obtain actual parameters of the obstacles and terrain; Compare the actual parameters of obstacles and terrain with the obstacles and terrain in the obstacle avoidance factors to determine the deviation points, where the deviation points include position and specification deviations, missing or newly added deviations; According to the deviation points, the obstacle avoidance factors are corrected on the ideal path.

4. The method for UAV route planning based on visual images according to claim 3 is characterized in that: After the obstacle avoidance factor on the ideal path is corrected, the correction result can be automatically memorized and applied when the route planning task is performed next time, and the frequency of UAV visual image acquisition can be increased before reaching the obstacle avoidance factor.

5. The method for UAV route planning based on visual images according to claim 1, characterized in that: The step of optimizing the ideal path based on the corrected obstacle avoidance factors and planning multiple executable paths includes: Based on the optimized ideal path, combined with the outline, size and location information of obstacles and terrain in the visual image information, the feasibility analysis of each ideal path is carried out, and the ideal path is classified into executable path and non-executable path; The non-executable paths are removed from the map, and only the executable paths are retained.

6. The method for UAV route planning based on visual images according to claim 5, characterized in that: The step of performing feasibility analysis on each ideal path and classifying the ideal path into executable paths and non-executable paths includes: Extracting density, height difference and flight time information of obstacles and terrain from each of the ideal paths; Evaluate the risk factor of each ideal path based on the density, altitude difference and flight time information; comparing the risk factor with a preset classification threshold; When the risk coefficient is lower than the classification threshold, it indicates that the ideal path corresponding to the risk coefficient has high safety, and the route corresponding to the ideal path is recorded as an executable path; When the risk coefficient is higher than or equal to the classification threshold, it indicates that the safety of the ideal path corresponding to the risk coefficient is low, and the route corresponding to the ideal path is recorded as a non-executable path.

7. The method for UAV route planning based on visual images according to claim 1, characterized in that: When the priority of each executable path is sorted according to the flight parameters, the risk coefficients under each executable path are arranged in descending order, and the executable path corresponding to the highest risk coefficient is determined as the preferred flight path, and the executable paths corresponding to the second highest and lower risk coefficients are determined as the alternative flight paths; Among them, the executable path with a smaller risk coefficient has a higher priority, and the executable path with a larger risk coefficient has a lower priority.

8. The method for UAV route planning based on visual images according to claim 7, characterized in that: The steps of selecting the drone route based on the priority sorting and adjusting the priority sorting according to the visual image information of the actual flight process include: The default preferred flight path is the drone route; Collect obstacle and terrain images captured by the drone in real time under the preferred flight path, and compare the obstacle and terrain images captured in real time with the planned preferred flight path; When the obstacle and terrain images captured in real time are not in the preferred flight path, a detour path is generated, and the degree of fit between the detour path and the preferred flight path and the alternative flight path is matched; The drone route is reselected based on the degree of adaptability. If the detour path has the highest degree of adaptability with the original preferred flight path, the flight continues according to the original detour path. Otherwise, the alternative flight path with the highest degree of adaptability is selected for flight, and the drone route planning is updated at the same time.

9. A UAV route planning system based on visual images, characterized in that: The method for UAV route planning based on visual images according to any one of claims 1 to 8 comprises: Obtain map data of the road section to be planned, plan an initial path in the map data in the form of the shortest path, optimize the initial path based on influencing factors to obtain multiple ideal paths, and mark obstacle avoidance factors in each ideal path, which include obstacles and terrain; Select multiple ideal paths as the planned routes of the drone, and turn on the automatic obstacle avoidance function of the drone to the maximum power consumption, so that the drone can obtain visual image information in the ideal path during flight, and use the obstacle images and terrain images captured during flight to determine the deviation points of obstacle avoidance factors based on the visual images, and correct the deviation points in the ideal path; Based on the corrected obstacle avoidance factors, the ideal path is optimized, multiple executable paths are planned, and the flight parameters of each executable path are collected. The executable paths are prioritized according to the flight parameters, where the flight parameters include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor energy consumption; Based on the priority ranking, the drone route is selected, and the priority ranking is adjusted according to the visual image information of the actual flight process.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the visual image-based drone route planning method described in any one of claims 1 to 8.

Images