Unmanned aerial vehicle obstacle avoidance path planning method, device, computer equipment and storage medium

Abstract

The present invention discloses a method, device, computer equipment and storage medium for planning a path for obstacle avoidance of a UAV. The method comprises obtaining a monitoring image stream of the airspace around the UAV, and judging whether there are dynamic obstacles around the UAV by a target detection algorithm; if there are, visually tracking the dynamic obstacles according to the detection results of the monitoring image stream to obtain a historical motion trajectory; predicting the motion trajectory of the dynamic obstacles based on the historical motion trajectory to obtain a predicted trajectory; obtaining the UAV's own motion information and current route information, and constructing a flight path for the UAV to avoid static obstacles according to a preset trajectory planning algorithm and the self-motion information and the current route information to obtain the planned path of the UAV; planning the speed of the UAV to avoid dynamic obstacles based on the planned path and the predicted trajectory to obtain the planned speed of the UAV. The method improves the flight stability and obstacle avoidance capability of fixed-wing UAVs.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment and storage medium for unmanned aerial vehicle obstacle avoidance path planning. Background Art

[0002] With the development of machine learning and deep learning, computer vision has made great progress in target recognition and environmental perception tasks. At present, the automotive and robotics industries have made extensive use of computer vision technology to improve their automation. At the same time, the aviation industry mainly relies on expensive radars and active communication ADS-B devices for target detection and obstacle avoidance tasks. However, with the widespread popularity of drones, the number of non-cooperative targets (targets that do not have active communication tools such as ADS-B) in the airspace in the future will increase greatly. How to detect these non-cooperative targets and make corresponding decisions to avoid collisions will be an essential function of future aircraft. Compared with expensive radars, cameras are cheap and can capture more visual information, and have the potential to replace radars on some low-cost aircraft. At present, there are technical applications that introduce binocular visual obstacle avoidance technology to consumer-grade rotorcraft. However, so far, there is no case of applying visual obstacle avoidance technology to fixed-wing aircraft that require a larger detection distance. Summary of the invention

[0003] The embodiments of the present invention provide a method, device, computer equipment and storage medium for planning an obstacle avoidance path for an unmanned aerial vehicle, aiming to solve the problem of poor visual obstacle avoidance effect of fixed-wing unmanned aerial vehicles in the prior art.

[0004] In a first aspect, an embodiment of the present invention provides a method for obstacle avoidance path planning for a drone, comprising:

[0005] Obtain a monitoring image stream of the airspace around the drone, detect the monitoring image stream using a target detection algorithm, and determine whether there are dynamic obstacles around the drone based on the detection results;

[0006] If so, visually track the dynamic obstacle according to the detection result of the monitoring image stream to obtain the historical movement trajectory of the dynamic obstacle;

[0007] Predicting the motion trajectory of the dynamic obstacle based on the historical motion trajectory to obtain a predicted trajectory of the dynamic obstacle, wherein the predicted trajectory includes the flight position, speed, acceleration, and attitude of the dynamic obstacle;

[0008] Acquire the self-motion information and current route information of the UAV, and construct a flight path for the UAV to avoid static obstacles according to a preset trajectory planning algorithm and the self-motion information and the current route information, so as to obtain a planned path for the UAV;

[0009] The speed of the UAV to avoid dynamic obstacles is planned based on the planned path and the predicted trajectory to obtain the planned speed of the UAV.

[0010] In a second aspect, an embodiment of the present invention provides a drone obstacle avoidance path planning device, which includes:

[0011] A detection module is used to obtain a monitoring image stream of the airspace around the drone, detect the monitoring image stream through a target detection algorithm, and determine whether there are dynamic obstacles around the drone based on the detection results;

[0012] A tracking module is used to visually track the dynamic obstacle when there is a dynamic obstacle around the drone according to the detection result of the monitoring image stream to obtain the historical movement trajectory of the dynamic obstacle;

[0013] A prediction module, used to predict the motion trajectory of the dynamic obstacle based on the historical motion trajectory to obtain a predicted trajectory of the dynamic obstacle, wherein the predicted trajectory includes the position, speed, acceleration and attitude of the dynamic obstacle;

[0014] A path planning module is used to obtain the self-motion information and current route information of the UAV, and construct a flight path for the UAV to avoid static obstacles according to a preset trajectory planning algorithm and the self-motion information and current route information, so as to obtain a planned path for the UAV;

[0015] The speed planning module is used to plan the speed of the UAV to avoid dynamic obstacles based on the planned path and the predicted trajectory to obtain the planned speed of the UAV.

[0016] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the drone obstacle avoidance path planning method described in the first aspect is implemented.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor enables the processor to execute the drone obstacle avoidance path planning method described in the first aspect above.

[0018] The embodiment of the present invention provides a method, device, computer equipment and storage medium for planning an obstacle avoidance path for a drone. The method includes obtaining a monitoring image stream of the airspace around the drone, detecting the monitoring image stream through a target detection algorithm, and judging whether there are dynamic obstacles around the drone according to the detection results; if there are, visually tracking the dynamic obstacles according to the detection results of the monitoring image stream to obtain the historical motion trajectory of the dynamic obstacles; predicting the motion trajectory of the dynamic obstacles based on the historical motion trajectory to obtain the predicted trajectory of the dynamic obstacles; obtaining the drone's own motion information and current route information, constructing a flight path for the drone to avoid static obstacles according to a preset trajectory planning algorithm and the self-motion information and the current route information to obtain the planned path of the drone; planning the speed of the drone to avoid dynamic obstacles based on the planned path and the predicted trajectory to obtain the planned speed of the drone. The method is applied to fixed-wing UAVs. The dynamic obstacles around the fixed-wing UAV are detected and tracked based on the target detection algorithm and visual tracking, so as to obtain the historical motion trajectory of the dynamic obstacles. The motion trajectory of the dynamic obstacles is then predicted based on the historical motion trajectory, so as to obtain the predicted trajectory of the dynamic obstacles. A planned path to avoid static obstacles is planned based on the fixed-wing UAV's own motion information and current route information. Then, the speed is dynamically adjusted in combination with the predicted trajectory of the dynamic obstacles and the planned path of the fixed-wing UAV itself, so as to avoid static obstacles while avoiding dynamic obstacles, effectively realize the route obstacle avoidance planning of the fixed-wing UAV, and improve the flight stability and obstacle avoidance capability of the fixed-wing UAV. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0020] Figure 1 A schematic diagram of a flow chart of a method for obstacle avoidance path planning for a drone provided in an embodiment of the present invention;

[0021] Figure 2 for Figure 1 A schematic diagram of a flow chart of an embodiment of step S130;

[0022] Figure 3 A schematic block diagram of a drone obstacle avoidance path planning device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0025] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] See also Figure 1 A schematic flow chart of a method for obstacle avoidance path planning for a UAV provided in an embodiment of the present invention, the method comprising steps S110 to S150.

[0028] Step S110: acquiring a monitoring image stream of the airspace around the drone, detecting the monitoring image stream using a target detection algorithm, and determining whether there are dynamic obstacles around the drone based on the detection results;

[0029] In this embodiment, the drone is a fixed-wing drone equipped with one or more cameras, and the camera collects the detection image stream of the airspace around the fixed-wing drone; wherein the camera is an RGB camera with a fixed-focus lens. The monitoring image stream is detected by a target detection algorithm, and it is determined whether there are dynamic obstacles around the drone based on the detection results. The dynamic obstacles include one or more aircraft other than the fixed-wing drone itself. Furthermore, the target detection algorithm used in this embodiment can be either a traditional HOG+SVM algorithm or a currently popular deep learning algorithm. The choice of a specific algorithm can be based on the size of the data set available for training and the computing power resources available when the device is actually running.

[0030] Step S120: If the obstacle exists, visually track the obstacle according to the detection result of the monitoring image stream to obtain the historical movement trajectory of the obstacle;

[0031] In this embodiment, if there is a dynamic obstacle in the detection result, a 2D target detection frame is generated around the dynamic obstacle, and the target detection frame includes the position and length and width information of the dynamic obstacle in the corresponding monitoring image; then the dynamic obstacle is visually tracked according to the detection result of the monitoring image stream to obtain the historical motion trajectory of the dynamic obstacle.

[0032] Furthermore, the methods of visually tracking dynamic obstacles according to the detection results of the monitoring image stream are divided into two categories:

[0033] The first category is Tracking by Detection. The specific process is: perform target detection on each frame of the monitoring image stream to obtain the detection result of each frame of the monitoring image (frame detection result); match the detection results (frame detection results) detected by two adjacent frames of monitoring images according to the relative position relationship and appearance similarity between the detection results (frame detection results) of the two adjacent frames of monitoring images to achieve tracking.

[0034] The second category is the template matching tracking method, and the specific process is: based on the frame detection results (frame detection results) of each frame of monitoring image, according to the detection results (frame detection results) of the previous frame of monitoring image, in the current frame, the pixels around the target detection box of each previous frame of monitoring image are matched and tracked.

[0035] Among them, Tracking by Detection can track multiple targets at the same time, while template matching tracking is only suitable for tracking a single or several targets. In actual applications, two visual tracking methods can be configured at the same time, and the corresponding method can be matched according to different needs for visual tracking. It should be noted that when performing visual tracking, a unique ID is configured for each tracking target and a 2D historical trajectory is generated for it (the 2D historical trajectory contains the pixel coordinate position of the tracking target in each frame).

[0036] Step S130: predicting the motion trajectory of the dynamic obstacle based on the historical motion trajectory to obtain a predicted trajectory of the dynamic obstacle, wherein the predicted trajectory includes the flight position, speed, acceleration and attitude of the dynamic obstacle;

[0037] In this embodiment, the motion trajectory of the dynamic obstacle is predicted based on the historical motion trajectory, specifically including: determining the first frame monitoring image of the dynamic obstacle based on the historical motion trajectory, extracting feature points of the pixel points in the target detection frame corresponding to the dynamic obstacle in the first frame monitoring image (i.e., in the 2D target detection frame), and obtaining the feature points of the dynamic obstacle, where the feature points can be SIFT feature points, FAST feature points, or ORB feature points; then obtaining the relative motion trajectory of the fixed-wing UAV itself and the feature point motion trajectory to triangulate the feature points of the dynamic obstacle, thereby obtaining the initial 3D coordinates of the dynamic obstacle in the 3D coordinate system, and then differentiating the initial 3D coordinates of the dynamic obstacle to obtain the 3D speed of the dynamic obstacle in the 3D coordinate system. Based on the initial 3D coordinates and 3D speed of the dynamic obstacle, combined with time, the future position of the dynamic obstacle can be predicted, thereby obtaining the predicted trajectory of the dynamic obstacle.

[0038] Furthermore, for the prediction of the future trajectory of the dynamic obstacle, the attitude and angular velocity of the dynamic obstacle flight must also be considered. The specific process of determining the attitude and angle of the dynamic obstacle flight includes: detecting the attitude of the dynamic obstacle in each frame of the monitoring image stream through a preset detection neural network, and calculating the angular velocity of the dynamic obstacle based on the attitude and corresponding time difference of two adjacent frames of monitoring images. Finally, the motion trajectory of the dynamic obstacle is predicted based on the initial 3D coordinates, 3D velocity, attitude and angular velocity to obtain the predicted trajectory of the dynamic obstacle.

[0039] like Figure 2 As shown, in one embodiment, step S130 further includes:

[0040] Step S210: identifying the dynamic obstacles in the current monitoring image based on the preset aircraft database, determining the type of the dynamic obstacles, and obtaining the real size of the dynamic obstacles;

[0041] Step S220: Compare the corresponding dynamic obstacles in the current monitoring image based on the real size to obtain a scaling ratio;

[0042] Step S230: Estimate the distance between the corresponding dynamic obstacle and the UAV in the current monitoring image according to the scaling ratio to obtain the real distance.

[0043] In this embodiment, in the process of predicting the motion trajectory of the dynamic obstacle based on the historical motion trajectory, the distance between the dynamic obstacle and the fixed-wing UAV is also estimated, which specifically includes: identifying the dynamic obstacle in the current monitoring image based on a preset aircraft database, determining the type of aircraft to which the dynamic obstacle belongs, and then determining the real size of the dynamic obstacle; then comparing the corresponding dynamic obstacle in the current monitoring image based on the real size of the dynamic obstacle, and determining the corresponding scaling ratio; finally, estimating the distance between the corresponding dynamic obstacle in the current monitoring image and the UAV according to the scaling ratio, and obtaining the real distance between the dynamic obstacle and the UAV.

[0044] For example, the type of aircraft to which the dynamic obstacle belongs is currently identified, and the scaling ratio between the actual size of the dynamic obstacle and the corresponding dynamic obstacle in the current monitoring image is determined to be 1:50. The distance between the corresponding dynamic obstacle and the fixed-wing UAV in the current monitoring image is 30 cm. In fact, the distance between the dynamic obstacle and the fixed-wing UAV is 1500 cm.

[0045] Step S140, obtaining the self-motion information and current route information of the UAV, constructing a flight path for the UAV to avoid static obstacles according to a preset trajectory planning algorithm and the self-motion information and the current route information, and obtaining a planned path for the UAV;

[0046] In this embodiment, based on the UAV's own motion information and current route information, a trajectory planning algorithm is used to construct a planned path that tracks the reference route and has no spatial and temporal intersection with dynamic obstacles. The construction of the planned path is divided into two steps: path decision and secondary planning. It is to determine a feasible path with the lowest cost near the reference route. The reference route is the route of the flight plan resident in the flight control system. There are many obstacle avoidance scenarios involved in the planned path, including whether the fixed-wing UAV should climb or descend, and whether it should detour left or right. The path decision process specifically includes: obtaining the UAV's own motion information and current route information, and determining the positions of static obstacles (such as mountains, buildings, etc.) that the fixed-wing UAV will encounter in the route based on the current route information and the preset terrain database; limiting the solution space of path planning and converting it into a convex problem. The main process is to convert the problem into a frenet coordinate system based on the reference route for research, and then construct multiple normal planes along the reference route currently corresponding to the fixed-wing UAV, and discretize the normal planes into cells (i.e., path cells), each of which represents the offset of the UAV relative to the reference route at the corresponding normal plane; connecting the path cells between adjacent normal planes based on the quintic polynomial and the position of static obstacles to obtain multiple candidate paths between adjacent normal planes; after obtaining all candidate paths, comprehensively consider the flight corridor, terrain, smoothness and static obstacle costs on the candidate paths, and calculate the cost C of each candidate path according to the following formula 成本 :

[0047] C 成本 =C1+C2+C3+C4,

[0048] Among them, C1 represents the cost of path smoothness, C2 represents the cost of static obstacles, C3 represents the cost of terrain, and C4 represents the cost of flight corridor;

[0049] Calculate C1 according to the following formula:

[0050] C1=ω1*∫l(s) 2 +ω2*∫l′(s) 2 +ω3*∫l″(s) 2 +ω4*∫g(s) 2 +ω5*∫g′(s) 2 +ω6*∫g″(s) 2 ,

[0051] Wherein, l(s) and g(s) represent the lateral offset and longitudinal offset on the normal plane, respectively; l′(s) and g′(s) represent the first-order derivatives of the lateral offset and longitudinal offset, respectively; l″(s) and g″(s) represent the second-order derivatives of the lateral offset and longitudinal offset, respectively; ω1, ω2, ω3, ω4, ω5, and ω6 represent weight coefficients;

[0052] Calculate C2, C3, and C4 according to the following formula:

[0053]

[0054] Where d represents the distance from the path point to the obstacle / terrain / flight corridor; d c ,d n represents the buffer distance; π represents a decreasing function. The closer the moving target is to the path point, the greater the path cost.

[0055] After obtaining the cost of each candidate path, the dynamic programming algorithm is used to find the candidate path with the lowest cost as the target path; finally, the target path is improved, and a convex space is generated using the dynamic programming path. The target path is smoothed through the convex space to obtain the final planned path.

[0056] Step S150: planning the speed of the UAV to avoid dynamic obstacles based on the planned path and the predicted trajectory to obtain the planned speed of the UAV.

[0057] In this embodiment, the main process of planning the speed includes: constructing an ST graph according to the predicted trajectory and planned path of the dynamic obstacle, in which the dynamic obstacle is represented in the form of a rectangle; discretizing the ST graph into a cell form (i.e., a speed cell), where the cell represents the position of the drone at time t, and each cell is used as a sampling point; connecting the cells at adjacent time points through a quintic polynomial to form a speed candidate configuration, counting all the connections to obtain a set of candidate speed configurations, and then comprehensively considering the speed requirement, smoothness, and the cost of the dynamic obstacle, and calculating the cost C of each candidate path according to the following formula: 成本 ′:

[0058] C 成本 ′=C′1+C′2+C5,

[0059] Where C1′ represents the cost of velocity smoothness, C2′ represents the cost of dynamic obstacles, and C5 represents the cost of velocity tracking;

[0060] Among them, the cost method of the dynamic obstacle C2′ is the same as that of the static obstacle C2;

[0061] Calculate C1′ and C5 according to the following formula:

[0062] C1=ω1*∫s″ 2 +ω2*∫s″′ 2 ,

[0063] C5=ω3*∫ρ(s′-V ref ) 2 ,

[0064] Where s″ and s″′ represent the acceleration and jerk along the planned path. When the speed s′ along the planned path deviates from the expected value V ref When , the ρ function increases the cost.

[0065] After obtaining the cost of each candidate speed configuration, the dynamic programming algorithm is used to find the candidate speed configuration with the lowest cost as the target speed configuration; finally, the target speed configuration is improved, and the convex space is generated by the dynamic planning path, and the target speed configuration is smoothed through the convex space to obtain the final planned speed. Finally, the fixed-wing UAV flies according to the planned path and planned speed to avoid static obstacles and dynamic obstacles.

[0066] The method is applied to fixed-wing UAVs. The dynamic obstacles around the fixed-wing UAV are detected and tracked based on the target detection algorithm and visual tracking, so as to obtain the historical motion trajectory of the dynamic obstacles. The motion trajectory of the dynamic obstacles is then predicted based on the historical motion trajectory, so as to obtain the predicted trajectory of the dynamic obstacles. A planned path to avoid static obstacles is planned based on the fixed-wing UAV's own motion information and current route information. Then, the speed is dynamically adjusted in combination with the predicted trajectory of the dynamic obstacles and the planned path of the fixed-wing UAV itself, so as to avoid static obstacles while avoiding dynamic obstacles, effectively realize the route obstacle avoidance planning of the fixed-wing UAV, and improve the flight stability and obstacle avoidance capability of the fixed-wing UAV.

[0067] The embodiment of the present invention also provides a drone obstacle avoidance path planning device, which is used to execute any embodiment of the drone obstacle avoidance path planning method described above. Figure 3 , Figure 3 1 is a schematic block diagram of a drone obstacle avoidance path planning device provided by an embodiment of the present invention. The drone obstacle avoidance path planning device 100 can be configured in a server.

[0068] like Figure 3 As shown, the drone obstacle avoidance path planning device 100 includes a detection module 110 , a tracking module 120 , a prediction module 130 , a path planning module 140 , and a speed planning module 150 .

[0069] The detection module 110 is used to obtain a monitoring image stream of the airspace around the drone, detect the monitoring image stream using a target detection algorithm, and determine whether there are dynamic obstacles around the drone based on the detection results;

[0070] The tracking module 120 is used to visually track the dynamic obstacle according to the detection result of the monitoring image stream when there is a dynamic obstacle around the drone, so as to obtain the historical movement trajectory of the dynamic obstacle;

[0071] A prediction module 130, configured to predict the motion trajectory of the dynamic obstacle based on the historical motion trajectory to obtain a predicted trajectory of the dynamic obstacle, wherein the predicted trajectory includes the position, speed, acceleration and attitude of the dynamic obstacle;

[0072] The path planning module 140 is used to obtain the self-motion information and current route information of the UAV, and construct a flight path for the UAV to avoid static obstacles according to a preset trajectory planning algorithm and the self-motion information and current route information, so as to obtain a planned path for the UAV;

[0073] The speed planning module 150 is used to plan the speed of the UAV to avoid dynamic obstacles based on the planned path and the predicted trajectory to obtain the planned speed of the UAV.

[0074] In one embodiment, the tracking module 120 includes:

[0075] A detection unit, used to perform target detection on each monitoring image frame in the monitoring image stream to obtain a frame detection result of each monitoring image frame;

[0076] The first matching unit is used to perform target matching and tracking according to the relative position relationship and appearance similarity of the frame detection results of two adjacent frames of monitoring images.

[0077] In one embodiment, the tracking module 120 includes:

[0078] The second matching unit is used to match and track pixels around each corresponding target detection frame in the current frame monitoring image according to the frame detection result of the previous frame monitoring image.

[0079] In one embodiment, the prediction module 130 further includes:

[0080] A feature point extraction unit is used to determine a first monitoring image of a dynamic obstacle based on the historical motion trajectory, extract feature points from pixels in a target detection frame corresponding to the dynamic obstacle in the first monitoring image, and obtain feature points of the dynamic obstacle;

[0081] A 3D velocity calculation unit is used to obtain the relative motion trajectory and the feature point motion trajectory of the drone itself, and triangulate the feature points of the dynamic obstacle according to the relative motion trajectory and the feature point motion trajectory of the drone itself to obtain the initial 3D coordinates of the dynamic obstacle in the 3D coordinate system, and to differentiate the initial 3D coordinates to obtain the 3D velocity of the dynamic obstacle in the 3D coordinate system;

[0082] an angular velocity calculation unit, configured to monitor the posture of the dynamic obstacle based on the monitoring image stream and a preset detection neural network, determine the posture of the dynamic obstacle in each frame of the detection image, and calculate the angular velocity of the dynamic obstacle based on the postures and corresponding time differences of two adjacent frames of monitoring images to obtain the angular velocity of the dynamic obstacle;

[0083] The prediction unit is used to predict the motion trajectory of the dynamic obstacle based on the initial 3D coordinates, 3D velocity, posture and angular velocity to obtain the predicted trajectory.

[0084] In one embodiment, the path planning module 140 includes:

[0085] A static obstacle determination unit, used to determine the position of static obstacles in the route of the UAV based on the current route information and a preset terrain database;

[0086] A first construction unit is used to construct a plurality of normal planes along the reference route currently corresponding to the UAV, and discretize the normal planes into path cells, wherein the path cells represent the offset of the UAV relative to the reference route at the corresponding normal plane;

[0087] A first connection unit, configured to connect path cells between adjacent normal planes based on a quintic polynomial and the position of the static obstacle to obtain a plurality of candidate paths between adjacent normal planes;

[0088] The first cost calculation unit is used to calculate the cost index of all candidate paths, and use a dynamic programming algorithm to screen out the target path with the lowest cost from all candidate paths and smooth it to obtain the planned path.

[0089] In one embodiment, the speed planning module 150 includes:

[0090] A second construction unit is used to construct an ST graph based on the planned path and the predicted trajectory, and discretize the ST graph into velocity cells, wherein the dynamic obstacles in the ST graph are constructed as rectangles, the rectangles represent obstacles in the planned path, and the velocity cells represent the position of the UAV at time t;

[0091] A second connection unit is used to connect the speed cells of adjacent time points based on the quintic polynomial and the position of the dynamic obstacle in the predicted trajectory to obtain a plurality of candidate speed configurations;

[0092] The second cost calculation unit is used to calculate the cost index of all the candidate speed configurations, and use a dynamic programming algorithm to screen out the target speed configuration with the lowest cost from all the candidate speed configurations and smooth it to obtain the planned speed.

[0093] An embodiment of the present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned drone obstacle avoidance path planning method when executing the computer program.

[0094] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to execute the above-described drone obstacle avoidance path planning method.

[0095] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the above-described equipment, devices and units can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0096] In the several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. Units with the same function may also be combined into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be an electrical, mechanical or other form of connection.

[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0098] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0099] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), disk or optical disk and other media that can store program codes.

[0100] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for unmanned aerial vehicle obstacle avoidance path planning, characterized in that: include: Obtain a monitoring image stream of the airspace around the drone, detect the monitoring image stream using a target detection algorithm, and determine whether there are dynamic obstacles around the drone based on the detection results; If so, visually track the dynamic obstacle according to the detection result of the monitoring image stream to obtain the historical movement trajectory of the dynamic obstacle; Predicting the motion trajectory of the dynamic obstacle based on the historical motion trajectory to obtain a predicted trajectory of the dynamic obstacle, wherein the predicted trajectory includes the flight position, speed, acceleration, and attitude of the dynamic obstacle; Acquire the self-motion information and current route information of the UAV, and construct a flight path for the UAV to avoid static obstacles according to a preset trajectory planning algorithm and the self-motion information and the current route information, so as to obtain a planned path for the UAV; Based on the planned path and the predicted trajectory, the speed of the UAV to avoid dynamic obstacles is planned to obtain the planned speed of the UAV; The step of predicting the motion trajectory of the dynamic obstacle based on the historical motion trajectory to obtain the predicted trajectory of the dynamic obstacle includes: Based on the preset aircraft database, the dynamic obstacles in the current monitoring image are identified, the types of dynamic obstacles are determined, and the real sizes of dynamic obstacles are obtained; Comparing the corresponding dynamic obstacles in the current monitoring image based on the real size to obtain a scaling ratio; The distance between the corresponding dynamic obstacle and the UAV in the current monitoring image is estimated according to the scaling ratio to obtain the real distance.

2. The method for obstacle avoidance path planning for a UAV according to claim 1, characterized in that: The visual tracking of the dynamic obstacle according to the detection result of the monitoring image stream includes: Performing target detection on each monitoring image frame in the monitoring image stream to obtain a frame detection result for each monitoring image frame; Target matching and tracking are performed based on the relative position relationship and appearance similarity of the frame detection results of two adjacent frames of monitoring images.

3. The method for obstacle avoidance path planning of a UAV according to claim 2, characterized in that: The performing target detection on each frame of the monitoring image stream to obtain a frame detection result for each frame of the image further includes: According to the frame detection result of the previous frame monitoring image, the pixels around each corresponding target detection frame in the current frame monitoring image are matched and tracked.

4. The method for obstacle avoidance path planning for a UAV according to claim 1, characterized in that: The step of predicting the motion trajectory of the dynamic obstacle based on the historical motion trajectory to obtain the predicted trajectory of the dynamic obstacle includes: Determine a first monitoring image of the dynamic obstacle based on the historical motion trajectory, extract feature points of pixels in a target detection frame corresponding to the dynamic obstacle in the first monitoring image, and obtain feature points of the dynamic obstacle; Obtain the relative motion trajectory and feature point motion trajectory of the drone itself, and triangulate the feature points of the dynamic obstacle according to the relative motion trajectory and feature point motion trajectory of the drone itself to obtain the initial 3D coordinates of the dynamic obstacle in the 3D coordinate system, and perform differentiation on the initial 3D coordinates to obtain the 3D velocity of the dynamic obstacle in the 3D coordinate system; Based on the monitoring image stream and the preset detection neural network, the posture of the dynamic obstacle is monitored, the posture of the dynamic obstacle in each frame of the detection image is determined, and the angular velocity of the dynamic obstacle is calculated based on the postures and corresponding time differences of two adjacent frames of monitoring images to obtain the angular velocity of the dynamic obstacle; The motion trajectory of the dynamic obstacle is predicted based on the initial 3D coordinates, 3D velocity, posture and angular velocity to obtain the predicted trajectory.

5. The method for obstacle avoidance path planning for a UAV according to claim 1, characterized in that: The flight path of the UAV avoiding static obstacles is constructed according to the preset trajectory planning algorithm, the self-motion information and the current route information to obtain the planned path of the UAV, including: Based on the current route information and the preset terrain database, determine the location of static obstacles in the UAV's route; Constructing multiple normal planes along the reference route currently corresponding to the UAV, and discretizing the normal planes into path cells, wherein the path cells represent the offset of the UAV relative to the reference route at the corresponding normal plane; Connecting path cells between adjacent normal planes based on a quintic polynomial and the position of the static obstacle to obtain multiple candidate paths between adjacent normal planes; Cost indicators are calculated for all candidate paths, and a target path with the lowest cost is selected from all candidate paths using a dynamic programming algorithm and smoothed to obtain the planned path.

6. The method for obstacle avoidance path planning for a UAV according to claim 1, characterized in that: The method of planning the speed of the UAV to avoid dynamic obstacles based on the planned path and the predicted trajectory to obtain the planned speed of the UAV includes: An ST graph is constructed based on the planned path and the predicted trajectory, and the ST graph is discretized into velocity cells, wherein the dynamic obstacles in the ST graph are constructed as rectangles, the rectangles represent obstacles in the planned path, and the velocity cells represent the position of the UAV at time t; Connecting speed cells at adjacent time points based on a quintic polynomial and positions of dynamic obstacles in the predicted trajectory to obtain a plurality of speed candidate configurations; Cost indicators are calculated for all candidate speed configurations, and a dynamic programming algorithm is used to screen out the target speed configuration with the lowest cost from all candidate speed configurations and smooth the configuration to obtain the planned speed.

7. A drone obstacle avoidance path planning device, characterized in that: include: A detection module is used to obtain a monitoring image stream of the airspace around the drone, detect the monitoring image stream through a target detection algorithm, and determine whether there are dynamic obstacles around the drone based on the detection results; A tracking module is used to visually track the dynamic obstacle when there is a dynamic obstacle around the drone according to the detection result of the monitoring image stream to obtain the historical movement trajectory of the dynamic obstacle; A prediction module, used to predict the motion trajectory of the dynamic obstacle based on the historical motion trajectory to obtain a predicted trajectory of the dynamic obstacle, wherein the predicted trajectory includes the position, speed, acceleration and attitude of the dynamic obstacle; A path planning module is used to obtain the self-motion information and current route information of the UAV, and construct a flight path for the UAV to avoid static obstacles according to a preset trajectory planning algorithm and the self-motion information and current route information, so as to obtain a planned path for the UAV; A speed planning module, used to plan the speed of the UAV to avoid dynamic obstacles based on the planned path and the predicted trajectory, and obtain the planned speed of the UAV; The step of predicting the motion trajectory of the dynamic obstacle based on the historical motion trajectory to obtain the predicted trajectory of the dynamic obstacle includes: Based on the preset aircraft database, the dynamic obstacles in the current monitoring image are identified, the types of dynamic obstacles are determined, and the real sizes of dynamic obstacles are obtained; Comparing the corresponding dynamic obstacles in the current monitoring image based on the real size to obtain a scaling ratio; The distance between the corresponding dynamic obstacle and the UAV in the current monitoring image is estimated according to the scaling ratio to obtain the real distance.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the unmanned aerial vehicle obstacle avoidance path planning method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to execute the drone obstacle avoidance path planning method as described in any one of claims 1 to 6.

Images