A Visual Image-Based Unmanned Aerial Vehicle (UAV) Flight Path Planning System and Method

Abstract

This invention belongs to the field of UAV flight path planning technology, specifically relating to a UAV flight path planning system and method based on visual images. This invention effectively avoids the inaccuracy problems caused by complex terrain and numerous obstacles in traditional methods by accurately planning UAV flight paths. By collecting and analyzing visual image information in real time, this invention can promptly detect and correct deviations in obstacle avoidance factors, thereby ensuring the safety and stability of the UAV during flight. Furthermore, this invention provides a scheme for optimizing the ideal path based on the corrected obstacle avoidance factors, further improving the feasibility and efficiency of flight path planning. In addition, through comprehensive evaluation of various flight parameters, this invention can scientifically prioritize various executable paths, thereby achieving intelligent selection of UAV flight paths. This not only improves the flight efficiency of the UAV but also reduces energy consumption during flight.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) flight path planning technology, specifically relating to a UAV flight path planning system and method based on visual images. Background Technology

[0002] Currently, when planning drone flight routes, the impact of obstacles on flight is generally taken into account. Therefore, obstacle avoidance is usually optimized during route planning. The conventional approach to obstacles is to choose to bypass them in order to reach the work point in time. However, in actual flight, the higher the obstacle avoidance accuracy, the greater the impact on the drone's endurance. Therefore, when planning drone flight routes, it is necessary to combine the actual path with the drone's endurance and plan the flight path in advance to reduce the high energy consumption caused by obstacle recognition and judgment during subsequent flight. This is also the mainstream planning method at present.

[0003] While the aforementioned methods can significantly improve the flight path planning capabilities of drones, the selection of flight paths by drones is generally based on flight distance. That is, when avoiding obstacles, the shorter path is prioritized, ignoring the actual operating status of the drone along each path, such as the drone's flight attitude and sensor operating frequency. This not only has a significant impact on the overall endurance of the drone, but also the possibility of dense obstacles in the shorter path. This undoubtedly increases the difficulty of the drone's flight mission. Furthermore, dense obstacles will also increase the drone's attitude changes and reduce its speed, which will shorten the actual endurance of the drone and affect actual operations. Summary of the Invention

[0004] The purpose of this invention is to provide a visual image-based UAV flight path planning system and method that can comprehensively consider various factors during UAV flight, such as density, altitude difference, flight time, path length, number of flight attitude adjustments, and obstacle avoidance difficulty, thereby planning a more reasonable and efficient flight path.

[0005] The specific technical solution adopted by this invention is as follows:

[0006] A method for UAV flight path planning based on visual images, comprising:

[0007] Obtain map data of the road segment to be planned, plan the initial path in the shortest path method in the map data, optimize the initial path based on influencing factors to obtain multiple ideal paths, and mark the obstacle avoidance factors in each ideal path. The obstacle avoidance factors include obstacles and terrain.

[0008] Multiple ideal paths are selected as the planned routes for the drone, and the drone's automatic obstacle avoidance function is turned on to the maximum power consumption. This allows the drone to acquire visual image information in the ideal paths during flight. The obstacle and terrain images captured during flight are used to determine the deviation points of obstacle avoidance factors based on the visual images, and the deviation points are corrected in the ideal paths.

[0009] Based on the corrected obstacle avoidance factors, the ideal path is optimized, multiple executable paths are planned, and the flight parameters of each executable path are collected. The executable paths are prioritized according to the flight parameters, which include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor power consumption.

[0010] The drone flight path is selected based on priority ranking, and the priority ranking is adjusted based on visual image information from the actual flight process.

[0011] In a preferred embodiment, the step of optimizing the initial path based on influencing factors to obtain multiple ideal paths includes:

[0012] Obtain geofence data for no-fly zones and mark it in the map data;

[0013] The portion of the initial path that crosses the no-fly zone is shifted to both sides to obtain multiple paths to be processed. When the initial path includes multiple no-fly zones and the distance between adjacent no-fly zones is less than a preset distance, the adjacent no-fly zones are merged and regarded as one no-fly zone.

[0014] Multiple paths to be processed are filtered out based on the drone's flight range. Paths with a length greater than the flight range are removed, as are paths that still cross the no-fly zone after deviating from the flight range.

[0015] The multiple paths to be processed after filtering are each denoted as ideal paths.

[0016] In a preferred embodiment, the step of determining obstacle avoidance factor deviation points based on the visual image and correcting the deviation points in the ideal path includes:

[0017] Obtain obstacle and terrain images from visual images, process the obstacle and terrain images, and obtain the actual parameters of obstacles and terrain;

[0018] The actual parameters of obstacles and terrain are compared with the obstacles and terrain in the obstacle avoidance factors to determine the deviation points, which include deviations in location and specifications, missing or added deviations.

[0019] Based on the deviation points, the obstacle avoidance factors are corrected on the ideal path.

[0020] In a preferred embodiment, after the obstacle avoidance factors on the ideal path are corrected, the corrected results can be automatically remembered and applied during the next execution of this route planning task, and the frequency of UAV visual image acquisition can be increased before reaching the obstacle avoidance factor.

[0021] In a preferred embodiment, the step of optimizing the ideal path and planning multiple executable paths based on the modified obstacle avoidance factors includes:

[0022] Based on the optimized ideal path, and combined with the outline, size and location information of obstacles and terrain in the visual image information, a feasibility analysis is performed on each ideal path, and the ideal paths are classified into executable paths and non-executable paths.

[0023] Remove the non-executable paths from the map, keeping only the executable paths.

[0024] In a preferred embodiment, the step of performing a feasibility analysis on each ideal path and classifying the ideal paths into executable paths and non-executable paths includes:

[0025] Extract obstacle and terrain density, height difference, and flight time information from each of the ideal paths;

[0026] Based on the density, altitude difference, and flight time information, assess the risk coefficient of each ideal path;

[0027] The risk coefficient is compared with a preset classification threshold;

[0028] When the risk coefficient is lower than the classification threshold, it indicates that the ideal path corresponding to the risk coefficient has high safety, and the route corresponding to this ideal path is recorded as an executable path.

[0029] When the risk coefficient is higher than or equal to the classification threshold, it indicates that the safety of the ideal path corresponding to the risk coefficient is low, and the route corresponding to this ideal path is recorded as an unexecutable path.

[0030] In a preferred embodiment, the step of selecting UAV flight routes based on priority ranking and adjusting the priority ranking according to visual image information from the actual flight process includes:

[0031] The default preferred flight path is the drone's flight path;

[0032] Real-time images of obstacles and terrain captured by the UAV along the preferred flight path are collected, and the real-time images of obstacles and terrain are compared with the planned preferred flight path.

[0033] When the real-time captured obstacle and terrain images are not in the preferred flight path, a detour path is generated, and the fit between the detour path and the preferred and alternative flight paths is matched.

[0034] The drone flight path is reselected based on the compatibility. If the detour path has the highest compatibility with the original preferred flight path, the drone will continue to fly along the original detour path. Otherwise, the alternative flight path with the highest compatibility will be selected, and the drone flight path plan will be updated.

[0035] The present invention also provides a visual image-based UAV flight path planning system using the above-described visual image-based UAV flight path planning method, comprising:

[0036] Obtain map data of the road segment to be planned, plan the initial path in the shortest path method in the map data, optimize the initial path based on influencing factors to obtain multiple ideal paths, and mark the obstacle avoidance factors in each ideal path. The obstacle avoidance factors include obstacles and terrain.

[0037] Multiple ideal paths are selected as the planned routes for the drone, and the drone's automatic obstacle avoidance function is turned on to the maximum power consumption. This allows the drone to acquire visual image information in the ideal paths during flight. The obstacle and terrain images captured during flight are used to determine the deviation points of obstacle avoidance factors based on the visual images, and the deviation points are corrected in the ideal paths.

[0038] Based on the corrected obstacle avoidance factors, the ideal path is optimized, multiple executable paths are planned, and the flight parameters of each executable path are collected. The executable paths are prioritized according to the flight parameters, which include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor power consumption.

[0039] The drone flight path is selected based on priority ranking, and the priority ranking is adjusted based on visual image information from the actual flight process.

[0040] And, an electronic device, the electronic device comprising:

[0041] At least one processor;

[0042] and a memory communicatively connected to the at least one processor;

[0043] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the above-described visual image-based UAV route planning method.

[0044] The technical effects achieved by this invention are as follows:

[0045] This invention effectively avoids the inaccuracy of flight path planning caused by complex terrain and numerous obstacles in traditional methods by precisely planning UAV flight paths. By collecting and analyzing visual image information in real time, this invention can promptly detect and correct deviations in obstacle avoidance factors, thereby ensuring the safety and stability of the UAV during flight. Furthermore, this invention provides a scheme for optimizing the ideal path based on the corrected obstacle avoidance factors, further improving the feasibility and efficiency of flight path planning. In addition, through a comprehensive evaluation of various flight parameters, this invention can scientifically prioritize various executable paths, thereby achieving intelligent selection of UAV flight paths. This not only improves the flight efficiency of the UAV but also reduces energy consumption during flight, providing strong technical support for the widespread application of UAVs.

[0046] This invention plans the flight path of a UAV, enabling the UAV to adjust its sensor power according to the position of obstacles and terrain along the planned flight path. When obstacles are dense, the obstacle avoidance capability is enhanced by increasing sensor power, while the power is reduced in other areas, thus ensuring the rational use of each sensor. In particular, when passing through deviation points, the obstacle avoidance capability is also appropriately increased to avoid temporary changes and enhance the safe flight capability of the UAV. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0048] Figure 2 This is a schematic diagram of the system modules of the present invention;

[0049] Figure 3 This is a schematic diagram of the electronic device structure of the present invention. Detailed Implementation

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0051] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0052] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments.

[0053] Please see Figure 1 As shown, this invention provides a method for UAV flight path planning based on visual images, including:

[0054] S1. Obtain map data of the road segment to be planned, plan the initial path in the shortest path in the map data, optimize the initial path based on influencing factors to obtain multiple ideal paths, and mark the obstacle avoidance factors in each ideal path. The obstacle avoidance factors include obstacles and terrain.

[0055] In step S1, when planning the flight path of the UAV, it is first necessary to obtain map data of the route segment to be planned. Then, an initial path is planned in the map data using the shortest path method. This initial path corresponds to the shortest physical distance and does not consider factors such as obstacles, terrain, and no-fly zones. Then, the initial path is optimized based on various influencing factors to obtain multiple ideal paths. In each ideal path, obstacle avoidance factors need to be marked. Obstacle avoidance factors mainly include obstacles and terrain. The steps of optimizing the initial path based on influencing factors to obtain multiple ideal paths include:

[0056] Obtain geofence data for no-fly zones and mark it in the map data;

[0057] The portion of the initial path that crosses the no-fly zone is shifted to both sides to obtain multiple paths to be processed. When the initial path includes multiple no-fly zones and the distance between adjacent no-fly zones is less than a preset distance, the adjacent no-fly zones are merged and regarded as one no-fly zone.

[0058] Multiple paths to be processed are filtered out based on the drone's flight range. Paths with a length greater than the flight range are removed, as are paths that still cross the no-fly zone after deviating from the flight range.

[0059] Each of the filtered paths to be processed is designated as an ideal path.

[0060] Specifically, when outputting the ideal path, the first step is to obtain geofence data for no-fly zones and accurately mark them on the map data to ensure that these areas can be accurately identified and avoided during path planning. Then, the initial path is analyzed. If the path crosses a no-fly zone, further adjustments are needed. Specifically, the portion of the initial path that crosses a no-fly zone is shifted to either side. This method yields multiple paths to be processed. When shifting the path, if the initial path contains multiple no-fly zones, and the distance between adjacent no-fly zones is less than a preset specific distance, then during the shifting process, it's necessary to simultaneously consider bypassing these adjacent no-fly zones to ensure the path's compliance and safety. For example, the distance between adjacent no-fly zones is 500m, and the preset specific distance is 100m. If the distance is 0m, then when offsetting the path, it is necessary to simultaneously bypass both no-fly zones (that is, treat two no-fly zones as one no-fly zone; of course, when more no-fly zones are adjacent to each other, treat multiple adjacent no-fly zones as one no-fly zone) to ensure that the drone does not enter the no-fly zone during flight. After obtaining multiple paths to be processed, further screening is required. The screening is based on the drone's range. Specifically, paths that exceed the drone's range will be eliminated, as these paths are impractical. Paths that will still cross other no-fly zones even after offsetting will also be eliminated. After this screening, the remaining paths to be processed can be identified as ideal paths. Ideal paths not only avoid no-fly zones but are also practically feasible flight routes within the drone's range.

[0061] S2. Select multiple ideal paths as the planned routes for the drone, and turn on the drone's automatic obstacle avoidance function to the maximum power consumption, so that the drone can acquire visual image information in the ideal path during flight. Use the obstacle images and terrain images captured during flight to determine the deviation points of obstacle avoidance factors based on the visual images, and correct the deviation points in the ideal path.

[0062] In step S2, after the ideal path is output, the drone's automatic obstacle avoidance function is activated to maximum power. This allows the drone to acquire real-time visual image information along the ideal path during flight, including obstacle and terrain images captured during flight. Based on the real-time visual image information, deviations in obstacle avoidance factors can be determined and corrected along the ideal path. Activating the drone's automatic obstacle avoidance function to maximum power ensures efficient obstacle avoidance during testing. In this state, sensors (such as radar) can accurately sense obstacles and terrain, making it easier for the drone to mark obstacles and terrain along the path during testing, in conjunction with visual image information. The steps of determining obstacle avoidance factor deviations based on visual images and correcting these deviations along the ideal path include:

[0063] Obtain obstacle and terrain images from visual images, process the obstacle and terrain images, and obtain the actual parameters of obstacles and terrain;

[0064] The actual parameters of obstacles and terrain are compared with the obstacles and terrain in the obstacle avoidance factors to determine the deviation points, which include deviations in location and specifications, missing or added deviations.

[0065] Based on the deviation points, the obstacle avoidance factors are corrected on the ideal path;

[0066] Specifically, when obstacle avoidance factors not marked in the ideal path appear in the real-time visual image information, the real-time obstacle position coordinates are first collected. Then compare the coordinates with the coordinates of the obstacle avoidance factors preset in the ideal path. The comparison is performed, and the initial distance deviation is output. The formula for calculating the initial distance deviation is: In the formula, This represents the initial distance deviation. Based on the initial distance deviation, the position and specification deviations, as well as the actual or newly added deviations, can be determined. The position and specification deviations are the differences between the actual position of the obstacle avoidance factor in the real-time image and the preset position in the ideal path. The missing deviation refers to the absence of the preset obstacle avoidance factor in the real-time image, while the newly added deviation refers to the appearance of an obstacle avoidance factor in the real-time image that was not preset in the ideal path. This allows for a more accurate determination of the actual impact of obstacle avoidance factors on the UAV's flight path, enabling corresponding corrections on the ideal path. The correction process includes the following two aspects:

[0067] For positional and specification deviations, a local path replanning approach is adopted to generate a detour trajectory based on the original ideal path, while satisfying the minimum turning radius constraint of the UAV:

[0068] ;

[0069] In the formula, Indicates the minimum turning radius. Indicates the drone's flight speed. Represents gravitational acceleration. Indicates the maximum roll angle;

[0070] For newly added obstacles, temporary waypoints need to be inserted into the path, and cubic spline interpolation is used to smoothly connect the old and new waypoints to ensure that the UAV can transition smoothly during flight and avoid flight instability or safety risks caused by sudden changes in flight direction.

[0071] In addition, for missing deviations, i.e. preset obstacle avoidance factors that do not appear in the real-time image, they will be reconfirmed to ensure that they have indeed disappeared due to changes in environmental factors. If they are confirmed to have disappeared, they will be deleted from the ideal path to avoid unnecessary flight adjustments. In this way, the UAV can make precise adjustments to its flight path based on real-time visual image information, thereby effectively avoiding obstacles and terrain obstacles and ensuring flight safety and efficiency.

[0072] Finally, the map data needs to be updated by using probability-based grid occupancy to update the obstacle position confidence level.

[0073] ;

[0074] In the formula, Indicates time All observation data Below is the entire environment map. The probability distribution, Represents the first in the raster map Units (each unit represents a spatial region). This represents the probability that a single grid cell is occupied by an obstacle;

[0075] Furthermore, after the obstacle avoidance factors on the ideal path are corrected, the correction results can be automatically remembered and applied when the same route planning task is executed next time. Before reaching the obstacle avoidance factor, the frequency of UAV visual image acquisition is increased to ensure more accurate identification and response to potential obstacles. If no corrected obstacle avoidance factors appear on the subsequent route, the frequency of UAV visual image acquisition is gradually reduced to save energy and improve flight efficiency. If corrected obstacle avoidance factors appear multiple times, they are recorded as regular obstacles and added to the map data. In subsequent route executions, these regular obstacles are treated as fixed obstacle avoidance factors, eliminating the need for real-time image acquisition and correction, thereby further optimizing the UAV's route planning.

[0076] S3. Based on the corrected obstacle avoidance factors, optimize the ideal path, plan multiple executable paths, and collect the flight parameters of each executable path. Prioritize each executable path according to the flight parameters. The flight parameters include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor power consumption.

[0077] In step S3, based on the corrected obstacle avoidance factors, the ideal path is further optimized to plan multiple executable paths. Subsequently, flight parameters of each executable path are collected, including path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor power consumption. Based on the flight parameters, the executable paths are prioritized. The step of optimizing the ideal path and planning multiple executable paths based on the corrected obstacle avoidance factors includes:

[0078] Based on the optimized ideal path, and combined with the outline, size and location information of obstacles and terrain in the visual image information, a feasibility analysis is performed on each ideal path, and the ideal paths are classified into executable paths and non-executable paths.

[0079] Remove non-executable paths from the map, keeping only executable paths;

[0080] Specifically, after considering the modified obstacle avoidance factors, the ideal path also needs to be optimized accordingly. First, based on the optimized ideal path, visual image information is further combined to conduct an in-depth analysis of the outline, size, and location information of obstacles and terrain. Based on this, a feasibility analysis can be performed on each ideal path, thus dividing the ideal paths into two categories: one is a path with the possibility of execution, and the other is a path without the conditions for execution. Finally, those paths that are determined to be non-executable will be removed from the map to ensure that only verified and feasible execution paths are retained on the map, so as to facilitate subsequent navigation and path planning work.

[0081] Secondly, a feasibility analysis is conducted on each ideal path, classifying the ideal paths into executable and non-executable paths, including the following steps:

[0082] Extract information on obstacle and terrain density, altitude difference, and flight time from each ideal path;

[0083] Based on density, altitude difference, and flight time information, assess the risk coefficient of each ideal path;

[0084] The risk coefficient is compared with a preset classification threshold;

[0085] When the risk coefficient is lower than the classification threshold, it indicates that the ideal path corresponding to the risk coefficient has high safety, and the route corresponding to this ideal path is recorded as an executable path.

[0086] When the risk coefficient is higher than or equal to the classification threshold, it indicates that the safety of the ideal path corresponding to the risk coefficient is low, and the route corresponding to this ideal path is recorded as an unexecutable path.

[0087] In this implementation, when dividing executable and non-executable paths, key information such as the density of obstacles and terrain, altitude difference, and flight time are first extracted from each ideal path.

[0088] Specifically, the density of obstacles and terrain is determined based on an obstacle density calculation function;

[0089] ;

[0090] In the formula, Indicates the density of obstacles and terrain. This indicates the total number of obstacles within the path area. Indicates the path length. This indicates the safe flight width of the drone (not less than twice the width of the fuselage).

[0091] The height difference evaluation function is:

[0092] ;

[0093] In the formula, Indicates the height difference. Indicates path sampling points altitude;

[0094] The flight time estimation function is:

[0095] ;

[0096] In the formula, This indicates the estimated flight time. This indicates the average flight speed of the drone. This indicates the number of times the drone's flight attitude was adjusted. This indicates the time taken for the drone to adjust its attitude during a single flight.

[0097] After outputting the obstacle density, flight altitude difference, and estimated flight time in the ideal path, these can be input together into a preset comprehensive evaluation function to assess the risk coefficient of each ideal path. The expression of the comprehensive evaluation function is as follows:

[0098] ;

[0099] In the formula, Indicates the risk coefficient. , and These represent the preset maximum density, altitude difference, and flight time (set according to mission requirements). , and These represent the weighting coefficients for obstacle density, altitude difference, and flight time, respectively. The calculated risk coefficients are then compared to pre-set classification thresholds. If the risk coefficient of an ideal path is lower than the threshold, it is considered to have high safety and is recorded as an executable path. Conversely, if the risk coefficient is higher than or equal to the threshold, the ideal path is considered less safe and is recorded as an unexecutable path. It should be further noted that when prioritizing executable paths based on flight parameters, the risk coefficients of each executable path are sorted in descending order. Among these, executable paths with lower risk coefficients have higher priority, while executable paths with higher risk coefficients have lower priority. When there are multiple executable paths with the same risk coefficient and the highest ranking, the path lengths are further compared, and the shortest executable path is determined as the highest priority path. If the path lengths are also the same, the number of flight attitude adjustments is compared, and the executable path with the fewest flight attitude adjustments is determined as the highest priority path. If the number of flight attitude adjustments is also the same, the sensor power consumption is compared, and the executable path with the lowest sensor power consumption is determined as the highest priority path. In this way, it can be ensured that the UAV selects the safest and most efficient flight path when performing a mission.

[0100] S4. Select the UAV route based on priority ranking, and adjust the priority ranking according to the visual image information of the actual flight process;

[0101] In step S4, after the priority ranking of executable paths is determined, a UAV flight path is selected based on the priority ranking. During actual flight, the priority ranking is dynamically adjusted based on real-time acquired visual image information to ensure that the UAV can safely complete its flight mission. The steps of selecting the UAV flight path based on the priority ranking and adjusting the priority ranking based on visual image information during actual flight include:

[0102] The default preferred flight path is the drone's flight path;

[0103] Real-time images of obstacles and terrain captured by the UAV along the preferred flight path are collected, and the real-time images of obstacles and terrain are compared with the planned preferred flight path.

[0104] When the real-time captured obstacle and terrain images are not in the preferred flight path, a detour path is generated, and the fit between the detour path and the preferred and alternative flight paths is matched.

[0105] The drone flight path is reselected based on the compatibility. If the detour path has the highest compatibility with the original preferred flight path, the drone will continue to fly along the original detour path. Otherwise, the alternative flight path with the highest compatibility will be selected and the drone flight path will be updated.

[0106] Specifically, based on a priority ranking mechanism, during actual flight, the predetermined priority ranking is dynamically adjusted according to the acquired visual image information. First, the default preferred flight path is used as the basic route for the UAV to perform its mission. Second, image information of various obstacles and terrain environments captured by the UAV during its flight along the preferred flight path is collected in real time and continuously. The real-time captured obstacle and terrain images are then compared and analyzed in detail with the pre-planned preferred flight path. If it is found that the real-time captured obstacle and terrain images are not included in the preset range of the original preferred flight path, an emergency plan is immediately activated to generate a corresponding detour path. There are multiple detour paths, and the detour ending position of each detour path coincides with both the original preferred flight path and the alternative flight path. To ensure flight safety and mission efficiency, the suitability of the detour path with the original preferred flight path and alternative flight paths is evaluated. First, the risk coefficient of the detour path needs to be determined (this is consistent with the method of determining the preferred flight path through the risk coefficient mentioned above, and will not be repeated here). This coefficient is recorded as the suitability. Next, based on the suitability results obtained from the evaluation, the flight path of the UAV is reselected. If the generated detour path has the highest suitability with the original preferred flight path, then the UAV continues to fly along that detour path. Otherwise, the alternative flight path with the highest suitability is selected to ensure that the UAV can complete the flight mission safely and efficiently. Finally, while completing the flight path selection, the flight path planning information of the UAV is updated simultaneously to optimize the overall execution effect of subsequent flight missions.

[0107] Please see Figure 2 A visual image-based UAV flight path planning system, using the aforementioned visual image-based UAV flight path planning method, includes:

[0108] The initialization module is used to acquire map data of the road segment to be planned, plan the initial path in the shortest path method in the map data, optimize the initial path based on influencing factors to obtain multiple ideal paths, and mark the obstacle avoidance factors in each ideal path. The obstacle avoidance factors include obstacles and terrain.

[0109] The deviation positioning module is used to select multiple ideal paths as the planned routes of the UAV and enable the UAV's automatic obstacle avoidance function to the maximum power consumption. This allows the UAV to acquire visual image information in the ideal path during flight. The module is used to capture obstacle and terrain images during flight, determine the deviation points of obstacle avoidance factors based on the visual images, and correct the deviation points in the ideal path.

[0110] The filtering and sorting module is used to optimize the ideal path based on the corrected obstacle avoidance factors, plan multiple executable paths, collect the flight parameters of each executable path, and sort the executable paths by priority according to the flight parameters. The flight parameters include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor power consumption.

[0111] The dynamic programming module is used to select UAV routes based on priority ranking and adjust the priority ranking according to visual image information during the actual flight process.

[0112] In the above, the main function of the initialization module is to acquire detailed map data of the road segment to be planned. After acquiring the map data, the system will initially plan an initial path based on the shortest path. Subsequently, the system will perform detailed optimization of this initial path based on various influencing factors, thereby generating multiple ideal paths. In the ideal paths, the system will further mark various obstacle avoidance factors, which include not only various obstacles but also complex terrain conditions. The role of the deviation positioning module is to select multiple optimized ideal paths as the planning routes of the UAV. During this process, the system will activate the UAV's automatic obstacle avoidance function to the maximum power consumption state to ensure that the UAV can efficiently acquire visual image information in the ideal paths during flight. The visual image information is mainly used to capture obstacle images and terrain images encountered during flight. The system will then use this visual image information to... The system accurately identifies deviations in obstacle avoidance factors and corrects these deviations in a timely manner along the ideal path. The filtering and sorting module further optimizes multiple ideal paths based on the corrected obstacle avoidance factors, thereby planning multiple executable paths. The system also collects detailed flight parameters for each executable path, including the total path length, the number of flight attitude adjustments, the difficulty of obstacle avoidance, and sensor power consumption. Based on these flight parameters, the system prioritizes each executable path to ensure that the UAV can choose the optimal path for flight. The dynamic planning module makes the final selection of the UAV's flight path based on the priority ranking results. During the actual flight of the UAV, the system dynamically adjusts the priority ranking based on real-time acquired visual image information to ensure that the UAV can flexibly respond to various emergencies and ensure flight safety and efficiency.

[0113] Please see Figure 3 An electronic device, comprising:

[0114] At least one processor;

[0115] and memory that is communicatively connected to at least one processor;

[0116] The memory stores a computer program that can be executed by at least one processor, which enables the at least one processor to execute the aforementioned visual image-based UAV route planning method.

[0117] The processor of the aforementioned electronic device can be

[0118] A central processing unit (CPU), graphics processing unit (GPU), or digital signal processor (DSP) may be used. Memory consists of external memory and internal memory. External memory includes any one of hard disk, solid-state drive (SSD), or flash memory. Internal memory includes any one of random access memory (RAM) or read-only memory (ROM). Electronic devices may also include an arithmetic logic unit (ALU), input devices, output devices, network interfaces, etc. The ALU may be used to perform various arithmetic and logical operations. Input devices may be keyboards, mice, touch screens, etc. Output devices may be monitors, printers, etc. Network interfaces may be Ethernet interfaces, wireless LAN interfaces, etc. Of course, electronic devices may also include other necessary components, such as power supplies and cooling fans.

[0119] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0120] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. A method for unmanned aerial vehicle (UAV) flight path planning based on visual images, characterized in that: This includes acquiring map data of the road segment to be planned, planning an initial path in the shortest path method from the map data, optimizing the initial path based on influencing factors to obtain multiple ideal paths, and marking obstacle avoidance factors in each ideal path, including obstacles and terrain; The steps for optimizing the initial path based on influencing factors to obtain multiple ideal paths include: acquiring geofence data for no-fly zones and marking it in map data; offsetting the portion of the initial path that crosses the no-fly zone to both sides to obtain multiple paths to be processed; when the initial path includes multiple no-fly zones and the distance between adjacent no-fly zones is less than a preset distance, the adjacent no-fly zones are merged and considered as one no-fly zone; filtering the multiple paths to be processed; based on the drone's flight range, removing paths whose length exceeds the flight range and paths that still cross no-fly zones after offset; and recording the filtered multiple paths to be processed as ideal paths. Multiple ideal paths are selected as the planned routes for the drone, and the drone's automatic obstacle avoidance function is enabled to maximum power consumption. This allows the drone to acquire visual image information along the ideal paths during flight. This information is used to capture obstacle and terrain images during flight. Based on the visual images, deviation points in obstacle avoidance factors are determined, and these deviations are corrected along the ideal paths. This process includes: acquiring obstacle and terrain images from the visual images; processing the obstacle and terrain images to obtain the actual parameters of the obstacles and terrain; comparing the actual parameters of the obstacles and terrain with the obstacle and terrain parameters in the obstacle avoidance factors to determine deviation points, including deviations in position and specifications, missing or added deviations; and correcting the obstacle avoidance factors along the ideal paths based on the deviation points. Based on the corrected obstacle avoidance factors, the ideal path is optimized, multiple executable paths are planned, and the flight parameters of each executable path are collected. The executable paths are prioritized according to the flight parameters, which include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor power consumption. The drone flight path is selected based on priority ranking, and the priority ranking is adjusted based on visual image information from the actual flight process.

2. The method for UAV route planning based on visual images according to claim 1, characterized in that: After the obstacle avoidance factors on the ideal path are corrected, the corrected results can be automatically remembered and applied when the same route planning task is executed next time, and the frequency of UAV visual image acquisition can be increased before reaching the obstacle avoidance factor.

3. The method for UAV flight path planning based on visual images according to claim 2, characterized in that: Based on the revised obstacle avoidance factors, the steps for optimizing the ideal path and planning multiple executable paths include: Based on the optimized ideal path, and combined with the outline, size and location information of obstacles and terrain in the visual image information, a feasibility analysis is performed on each ideal path, and the ideal paths are classified into executable paths and non-executable paths. Remove non-executable paths from the map, keeping only executable paths.

4. The UAV flight path planning method based on visual images according to claim 3, characterized in that: The steps involved in conducting a feasibility analysis for each ideal path and classifying them into executable and non-executable paths include: Extract information on obstacle and terrain density, altitude difference, and flight time from each ideal path; Based on density, altitude difference, and flight time information, assess the risk coefficient of each ideal path; The risk coefficient is compared with a preset classification threshold; When the risk coefficient is lower than the classification threshold, it indicates that the ideal path corresponding to the risk coefficient has high safety, and the route corresponding to this ideal path is recorded as an executable path. When the risk coefficient is higher than or equal to the classification threshold, it indicates that the safety of the ideal path corresponding to the risk coefficient is low, and the route corresponding to this ideal path is recorded as an unexecutable path.

5. The UAV flight path planning method based on visual images according to claim 4, characterized in that: When prioritizing each executable path based on flight parameters, the risk coefficients of each executable path are sorted in descending order, and the executable path with the highest risk coefficient is determined as the preferred flight path, while the executable paths with the second highest and lower risk coefficients are determined as alternative flight paths. Among them, the executable path with the lower risk coefficient has a higher priority, and the executable path with the higher risk coefficient has a lower priority.

6. The method for UAV flight path planning based on visual images according to claim 5, characterized in that: The steps of selecting drone flight routes based on priority ranking and adjusting the priority ranking based on visual image information from the actual flight process include: The default preferred flight path is the drone's flight path; Real-time images of obstacles and terrain captured by the UAV along the preferred flight path are collected, and the real-time images of obstacles and terrain are compared with the planned preferred flight path. When the real-time captured obstacle and terrain images are not in the preferred flight path, a detour path is generated, and the fit between the detour path and the preferred and alternative flight paths is matched. The drone flight path is reselected based on the compatibility. If the detour path has the highest compatibility with the original preferred flight path, the drone will continue to fly along the original detour path. Otherwise, the alternative flight path with the highest compatibility will be selected, and the drone flight path plan will be updated.

7. A visual image-based unmanned aerial vehicle (UAV) flight path planning system, characterized in that: The UAV route planning method based on visual images according to any one of claims 1 to 6 includes: Obtain map data of the road segment to be planned, plan the initial path in the shortest path method in the map data, optimize the initial path based on influencing factors to obtain multiple ideal paths, and mark the obstacle avoidance factors in each ideal path. The obstacle avoidance factors include obstacles and terrain. Multiple ideal paths are selected as the planned routes for the drone, and the drone's automatic obstacle avoidance function is turned on to the maximum power consumption. This allows the drone to acquire visual image information in the ideal paths during flight. The obstacle and terrain images captured during flight are used to determine the deviation points of obstacle avoidance factors based on the visual images, and the deviation points are corrected in the ideal paths. Based on the corrected obstacle avoidance factors, the ideal path is optimized, multiple executable paths are planned, and the flight parameters of each executable path are collected. The executable paths are prioritized according to the flight parameters, which include path length, number of flight attitude adjustments, obstacle avoidance difficulty, and sensor power consumption. The drone flight path is selected based on priority ranking, and the priority ranking is adjusted based on visual image information from the actual flight process.

8. An electronic device, characterized in that: Electronic devices include: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores a computer program that can be executed by at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the visual image-based UAV route planning method according to any one of claims 1 to 6.

Images