Unmanned aerial vehicle obstacle avoidance method and device, unmanned aerial vehicle and storage medium

Abstract

The invention relates to the technical field of unmanned aerial vehicles. The invention discloses an unmanned aerial vehicle obstacle avoidance method, an unmanned aerial vehicle obstacle avoidance device, an unmanned aerial vehicle and a storage medium, which can improve the planning speed of a flight path and timely respond to environmental changes so as to ensure the flight safety of the unmanned aerial vehicle. The unmanned aerial vehicle obstacle avoidance method comprises the steps of obtaining first point cloud data; inputting the first point cloud data into a positioning mapping module of an autonomous exploration framework for data analysis, and outputting real-time positioning information and second point cloud data; inputting the second point cloud data into a point cloud segmentation module of the autonomous exploration framework for segmentation processing, and outputting to-be-processed point cloud information; inputting the to-be-processed point cloud information into a recognition frame penetrating module of the autonomous exploration framework to perform structural feature matching processing, and outputting task point information; inputting the task point information and the real-time positioning information into a local planning module of an autonomous exploration framework for local path planning, and outputting a first flight path for obstacle avoidance; and executing the flight task based on the first flight path.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and more specifically to a method and device for avoiding obstacles in unmanned aerial vehicles, an unmanned aerial vehicle and a storage medium. Background Art

[0002] Traditional obstacle avoidance methods for drones usually use the A* algorithm (A-star Algorithm) for path planning. However, during the operation of the A* algorithm, each step needs to calculate the cost of the current node to determine the next search direction. When the number of nodes increases, the amount of calculation will increase significantly, resulting in longer algorithm running time. In the application scenario of drone path planning, the increase in the amount of calculation will cause the algorithm to be unable to respond to environmental changes in a timely manner, thereby affecting the flight safety of the drone. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method and device for avoiding obstacles in a drone, a drone and a storage medium, which can improve the flight path and ensure the flight safety of the drone. The embodiments of the present application are mainly implemented through the following technical solutions:

[0004] A first aspect of an embodiment of the present application provides a method for avoiding obstacles in a drone, comprising:

[0005] Acquire the first point cloud data;

[0006] Input the first point cloud data into the positioning and mapping module of the autonomous exploration framework for data analysis, and output real-time positioning information and second point cloud data;

[0007] Inputting the second point cloud data into the point cloud segmentation module of the autonomous exploration framework for segmentation processing, and outputting the point cloud information to be processed;

[0008] Input the point cloud information to be processed into the recognition and penetration module of the autonomous exploration framework for structural feature matching processing, and output the task point information;

[0009] Inputting the mission point information and the real-time positioning information into a local planning module of the autonomous exploration framework for local path planning, and outputting a first flight path for obstacle avoidance;

[0010] A flight mission is performed based on the first flight path.

[0011] According to an embodiment of the present application, the step of inputting the first point cloud data into the positioning and mapping module of the autonomous exploration framework for data analysis and outputting real-time positioning information and second point cloud data includes:

[0012] Performing denoising and dedistortion processing on the first point cloud data to obtain the second point cloud data;

[0013] The first point cloud data is input into the extended Kalman filter of the positioning and mapping module, and the extended Kalman filter fuses the first point cloud data with the first IMU data to obtain the real-time positioning information.

[0014] According to an embodiment of the present application, the step of inputting the real-time second point cloud data into the point cloud segmentation module of the autonomous exploration framework for segmentation processing and outputting the point cloud information to be processed includes:

[0015] The point cloud segmentation module calculates the normal Z value corresponding to each point in the second point cloud data by using a surface normal estimation method;

[0016] The normal Z value corresponding to each point in the second point cloud data is threshold filtered to obtain non-ground point cloud information, and the non-ground point cloud information is input into the RANSAC algorithm of the point cloud segmentation module. The RANSAC algorithm analyzes the non-ground point cloud information, excludes the noise point cloud information in the non-ground point cloud information, fits the plane model of the ground, and segments the ground point cloud from the second point cloud data to obtain the point cloud information to be processed.

[0017] According to an embodiment of the present application, the step of inputting the point cloud information to be processed into the recognition and penetration module of the autonomous exploration framework for structural feature matching processing and outputting the task point information includes:

[0018] Selecting one or more points from the point cloud information to be processed as initial seed points;

[0019] The points adjacent to each initial seed point are classified by combining the Euclidean clustering algorithm and the neighboring search algorithm to obtain the cluster corresponding to each initial seed point.

[0020] The structural features of all the clusters are matched with preset structural features to obtain the task point information.

[0021] According to one embodiment of the present application, after the step of performing the flight mission based on the first flight path, the unmanned aerial vehicle obstacle avoidance method further includes:

[0022] Acquire a first ground image;

[0023] The target detection module of the autonomous exploration framework is used to analyze and process the first ground image to obtain first target point information;

[0024] Modify the first flight path based on the first target point information to obtain a second flight path;

[0025] The flight attitude of the UAV is adjusted based on the second flight path.

[0026] According to one embodiment of the present application, after the step of adjusting the flight attitude of the drone based on the second flight path, the drone obstacle avoidance method further includes:

[0027] When the first target point information is landing point information, the landing module of the autonomous exploration framework modifies the second flight path based on the landing point information to obtain a third flight path;

[0028] The flight attitude of the UAV is adjusted based on the third flight path to perform a landing mission.

[0029] According to one embodiment of the present application, after the step of adjusting the flight attitude of the drone based on the third flight path and performing a landing task, the drone obstacle avoidance method further includes:

[0030] acquiring a second ground image;

[0031] modifying the third flight path based on the second ground image to obtain a fourth flight path;

[0032] The flight attitude of the UAV is adjusted based on the fourth flight path.

[0033] A second aspect of the embodiment of the present application provides a drone obstacle avoidance device, comprising:

[0034] An acquisition module, used for acquiring first point cloud data;

[0035] A real-time positioning information output module, used to input the first point cloud data into the positioning and mapping module of the autonomous exploration framework for data analysis, and output real-time positioning information and second point cloud data;

[0036] A point cloud information output module to be processed, used for inputting the second point cloud data into the point cloud segmentation module of the autonomous exploration framework for segmentation processing, and outputting the point cloud information to be processed;

[0037] A task point information output module is used to input the to-be-processed point cloud information into the recognition and penetration module of the autonomous exploration framework for structural feature matching processing and output task point information;

[0038] A first flight path output module, used for inputting the mission point information and the real-time positioning information into the local planning module of the autonomous exploration framework for local path planning, and outputting a first flight path for obstacle avoidance;

[0039] A flight mission execution module is used to execute the flight mission based on the first flight path.

[0040] According to a third aspect of an embodiment of the present application, a drone is provided, comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the steps of the drone obstacle avoidance method described in the first aspect of the embodiment of the present application.

[0041] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, and the computer program enables a computer to execute the steps of the drone obstacle avoidance method described in the first aspect of the embodiment of the present application.

[0042] The beneficial effects of the embodiments of the present application include:

[0043] The embodiment of the present application designs an autonomous exploration framework, which uses a screening method of structural feature matching to obtain task point information to generate a flight path. Specifically, the embodiment of the present application obtains the first point cloud data; inputs the first point cloud data into the positioning and mapping module of the autonomous exploration framework for data analysis, and outputs real-time positioning information and second point cloud data; inputs the second point cloud data into the point cloud segmentation module of the autonomous exploration framework for segmentation processing, and outputs the point cloud information to be processed; inputs the point cloud information to be processed into the identification and penetration module of the autonomous exploration framework for structural feature matching processing, and outputs the task point information; inputs the task point information and the real-time positioning information into the local planning module of the autonomous exploration framework for local path planning, and outputs the first flight path for obstacle avoidance; and executes the flight mission based on the first flight path. The present application only needs to screen out the task point information that matches the structural features to perform the next step of calculation operations, and does not need to process the task point information that does not match the structural features. Therefore, compared with the prior art, the embodiment of the present application can improve the planning speed of the flight path, respond to environmental changes in a timely manner, and thus ensure the flight safety of the drone.

[0044] The embodiment of the present application acquires a first ground image and uses a target detection module of an autonomous exploration framework to analyze and process the first ground image, thereby being able to adjust the flight path of the drone in real time in a dynamic environment to achieve dynamic obstacle avoidance.

[0045] The drone obstacle avoidance method provided in the embodiment of the present application adjusts the landing path in real time during the landing process by calculating the distance information between the center of the pixel in the second ground image and the center of the downward-looking camera of the drone in the x-axis and y-axis directions, thereby ensuring that the drone can land smoothly. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 A flowchart of the obstacle avoidance method for a drone of the present invention in some embodiments;

[0048] Figure 2 A flowchart of the autonomous exploration framework of the present invention in some embodiments;

[0049] Figure 3 is a block diagram of the obstacle avoidance device for a drone of the present invention in some embodiments;

[0050] Figure 4 A block diagram of a drone of the present invention in some embodiments. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0052] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0053] The terms "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0054] The terms "comprises", "includes" or any other variants thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. Unless otherwise defined, all technical and scientific terms used in the specification of the present application have the same meaning as those commonly understood by technicians in the technical field of the present application. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The term "and / or" used in the specification of the present application includes any and all combinations of one or more related listed items.

[0055] Autonomous navigation algorithms used in UAVs, such as the A* algorithm, the RRT algorithm, and traditional path planning methods, can effectively generate paths, but they are still insufficient in terms of dynamic environment adaptability, real-time performance, and multi-task processing capabilities. Therefore, the inventors have designed an autonomous exploration framework, which includes at least a local planning module, which can achieve efficient and real-time path planning in complex environments, and has good dynamic obstacle avoidance and multi-target tracking capabilities.

[0056] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0057] <Drone obstacle avoidance method>

[0058] like Figure 1 FIG. 1 is a flowchart of a method for avoiding obstacles in a drone provided by the first aspect of the embodiment of the present application. Figure 1 In the method, the obstacle avoidance method of the UAV includes:

[0059] S1. Obtain the first point cloud data.

[0060] In the embodiment of the present application, the drone is equipped with a high-precision laser radar, which can collect point cloud data of the drone's surrounding environment in real time during flight. The first point cloud data is the point cloud data collected by the laser radar.

[0061] Compared with the existing technology that uses images to identify specific targets or objects, the present application uses point cloud data, which can not only avoid the influence of lighting changes and occlusion by other objects, but also has sufficient and significant spatial depth information.

[0062] S2. Input the first point cloud data into the positioning and mapping module of the autonomous exploration framework for data analysis, and output real-time positioning information and second point cloud data.

[0063] In the embodiments of the present application, reference is made to Figure 2 The autonomous exploration framework includes a positioning and mapping module, a point cloud segmentation module, a frame recognition module, and a local planning module (i.e. Figure 2 local planner in .

[0064] Furthermore, the S2 step includes:

[0065] S21. Perform denoising and dedistortion processing on the first point cloud data to obtain the second point cloud data.

[0066] Specifically, in the embodiment of the present application, noise points are removed by a filter, and distortion in the first point cloud data is removed by a calibration algorithm to obtain the second point cloud data. That is, the second point cloud data is the point cloud filtered by the positioning and mapping module.

[0067] Step S21 performs data processing on the first point cloud data to ensure the accuracy and reliability of the second point cloud data.

[0068] S22. Input the first point cloud data into the extended Kalman filter of the positioning and mapping module, and the extended Kalman filter fuses the first point cloud data with the first IMU data to obtain the real-time positioning information.

[0069] The Extended Kalman Filter (EKF) is a highly efficient recursive filter (autoregressive filter) that can estimate the state of a dynamic system from a series of measurements that do not completely contain noise.

[0070] The first IMU data is the data of the Inertial Measurement Unit (IMU), which usually includes the readings of the accelerometer and the gyroscope, and is used to measure the acceleration and angular velocity of the object.

[0071] The embodiment of the present application uses the extended Kalman filter to fuse the first point cloud data and the first IMU data, which can continuously correct the positioning error through prediction and update steps, ensure the stable positioning of the drone in a dynamic environment, and thus improve the accuracy of the real-time positioning information.

[0072] It should be understood that in the autonomous positioning technology of drones, GPS, visual odometer VO and laser-inertial odometer LIO are the most commonly used positioning technologies. Among them, GPS determines the position by receiving satellite signals. However, GPS performs poorly in terms of positioning accuracy, reliability and accuracy indoors and in electromagnetic interference environments. The visual odometer VO uses a camera to capture image sequences and estimates the movement of the camera through image feature matching. However, this positioning technology is easily affected by lighting changes and dynamic environments. The laser-inertial odometer LIO achieves real-time, robust, and high-precision positioning and map construction in various environments by integrating the high-precision distance measurement of the lidar and the inertial data of the IMU. The embodiment of the present application uses the laser-inertial odometer LIO to achieve precise positioning.

[0073] S3. Input the second point cloud data into the point cloud segmentation module of the autonomous exploration framework for segmentation processing, and output the point cloud information to be processed.

[0074] Furthermore, the S3 step includes:

[0075] S31. The point cloud segmentation module uses a surface normal estimation method to calculate the normal Z value corresponding to each point in the second point cloud data.

[0076] The surface normal estimation method is used to estimate the normal direction of each point in the real-time positioning information, so as to better understand the geometric structure of the real-time positioning information.

[0077] The normal Z value represents the vertical direction of the normal.

[0078] S32. Perform threshold filtering on the normal Z value corresponding to each point in the second point cloud data to obtain non-ground point cloud information, and input the non-ground point cloud information into the RANSAC algorithm of the point cloud segmentation module. The RANSAC algorithm analyzes the non-ground point cloud information, excludes the noise point cloud information in the non-ground point cloud information, fits the plane model of the ground, and segments the ground point cloud from the second point cloud data to obtain the point cloud information to be processed.

[0079] It should be understood that step S32 is used to remove the ground point cloud in the second point cloud data.

[0080] The point cloud information to be processed is the environment point cloud remaining after the ground point cloud is removed from the second point cloud data.

[0081] Step S32 can achieve the segmentation of the ground point cloud, ensure the accurate extraction of the ground point cloud, and lay the foundation for the subsequent task point separation.

[0082] S4. Input the point cloud information to be processed into the recognition and penetration module of the autonomous exploration framework for structural feature matching processing, and output the task point information.

[0083] Furthermore, the S4 step includes:

[0084] S41, selecting one or more points from the point cloud information to be processed as initial seed points.

[0085] S42. Classify the points adjacent to each initial seed point by combining the Euclidean clustering algorithm and the neighboring search algorithm to obtain a cluster corresponding to each initial seed point.

[0086] The embodiment of the present application uses the Euclidean clustering algorithm to segment the point cloud information to be processed, and then uses a neighbor search algorithm to cluster points that are close to each other, so that different structural features can be identified.

[0087] S43: Match the structural features of all the clusters with preset structural features to obtain the task point information.

[0088] The preset structural feature is a frame and / or a ring structure. In other embodiments, the preset structural feature can be set by those skilled in the art according to actual needs. The task point information is the point cloud that meets the preset structural feature in the point cloud information to be processed.

[0089] The implementation of steps S41 to S43 can separate mission points with obvious structural features such as frames and rings (i.e., ring structures), so as to improve the accuracy of the first flight path and realize the autonomous obstacle avoidance function of the drone.

[0090] S5. Input the mission point information and the real-time positioning information into the local planning module of the autonomous exploration framework to perform local path planning, and output a first flight path for obstacle avoidance.

[0091] The S5 step includes:

[0092] S51, generating an initial trajectory from the starting point to the end point according to the mission point information and the real-time positioning information.

[0093] It should be understood that the initial trajectory is a B-spline curve Φ that only satisfies the terminal (drone) constraints without considering obstacle avoidance.

[0094] S52: Perform trajectory collision detection on the initial trajectory to obtain a detection result.

[0095] The detection principle is to first set the initial trajectory segment to multiple control points {a 1 , a 2 , a3 , …, a 20}, then the A* algorithm is used to construct the multiple control points into a first safe trajectory τ, passing through the control points {a 1 , a 2 , a 3 , …, a 20 The point where the perpendicular normal plane of the tangent vector of} intersects with the first safety trajectory τ intersects at the obstacle surface point {β 1 , β 2 , β 3 , …, β 20}, and obtain the vector v ij , v ij is a i Pointing to beta j The unit vector of the control point is used to determine whether it is within the obstacle according to the distance field definition formula. The distance field definition formula is:

[0096] d ij =(a i -β j )×v ij ;

[0097] Among them, a i is the i-th control point; β j For a i The corresponding obstacle surface point; v ij for a i Pointing to beta j The unit vector of .

[0098] The d ij That is the detection result.

[0099] S53: When the detection result is within the obstacle, adjust the initial trajectory to obtain a second safe trajectory.

[0100] The second safe trajectory is a safe and collision-free trajectory.

[0101] S54: Optimize the second safety trajectory to obtain the first flight path.

[0102] Specifically, the smoothness and minimum loss of the second safety trajectory are optimized, and the specific optimization formula is (that is, the optimization problem of the second safety trajectory is defined as follows in this embodiment of the application):

[0103] min a J=γ s J s +γ c j c +γ d J d;

[0104] Among them, J s As a smoothness penalty, the sum of squares of acceleration and jerk is reduced by minimizing the sum of squares of the second-order and third-order control points of the second safety trajectory; J c It is a collision penalty to keep the control point away from obstacles, which is achieved through safety gap and penalty function; d is the dynamic feasibility penalty, which limits the high-order derivatives of the second safety trajectory in each dimension to ensure feasibility; γ s For J s The weight of c For J c The weight of d For J d The safety gap can be set by those skilled in the art according to actual needs.

[0105] The quasi-Newton method is used to approximate the Hessian matrix from the gradient information to speed up the convergence.

[0106] It should be understood that during the execution of the second safety trajectory, the optimization formula is continuously changed as new obstacles are added, and the solver quickly restarts and solves. The planning algorithm comprehensively considers the flight dynamics characteristics and safety requirements of the UAV to ensure that the UAV can safely avoid obstacles in a complex environment and smoothly reach the mission point (that is, any point in the mission point information).

[0107] S6. Execute a flight mission based on the first flight path.

[0108] The embodiment of the present application designs an autonomous exploration framework, which uses a screening method of structural feature matching to obtain task point information to generate a flight path. Specifically, the embodiment of the present application obtains the first point cloud data; inputs the first point cloud data into the positioning and mapping module of the autonomous exploration framework for data analysis, and outputs real-time positioning information and second point cloud data; inputs the second point cloud data into the point cloud segmentation module of the autonomous exploration framework for segmentation processing, and outputs the point cloud information to be processed; inputs the point cloud information to be processed into the identification and penetration module of the autonomous exploration framework for structural feature matching processing, and outputs the task point information; inputs the task point information and the real-time positioning information into the local planning module of the autonomous exploration framework for local path planning, and outputs the first flight path for obstacle avoidance; and executes the flight mission based on the first flight path. The present application only needs to screen out the task point information that matches the structural features to perform the next step of calculation operations, and does not need to process the task point information that does not match the structural features. Therefore, compared with the prior art, the embodiment of the present application can improve the planning speed of the flight path, respond to environmental changes in a timely manner, and thus ensure the flight safety of the drone.

[0109] In some embodiments, after step S6, the drone obstacle avoidance method further includes:

[0110] S71. Acquire a first ground image.

[0111] The drone is also equipped with a downward-looking camera for collecting the first ground image in real time. The first ground image is an RGB image.

[0112] S72. Use the target detection module of the autonomous exploration framework to analyze and process the first ground image to obtain first target point information.

[0113] refer to Figure 2 As shown, the autonomous exploration framework also includes a target detection module.

[0114] The target detection module uses a target detection algorithm to analyze and process the first ground image. More specifically, the target detection algorithm is a Yolo (You Only Look Once) target detection algorithm.

[0115] The first target point information is a target object, which may be a building, a vehicle, a crop, a mountain, or a river.

[0116] S73. Modify the first flight path based on the first target point information to obtain a second flight path.

[0117] S74. Adjust the flight attitude of the UAV based on the second flight path.

[0118] The embodiment of the present application acquires a first ground image and uses a target detection module of an autonomous exploration framework to analyze and process the first ground image, thereby being able to adjust the flight path of the drone in real time in a dynamic environment to achieve dynamic obstacle avoidance.

[0119] In some embodiments, after the step S74, the drone obstacle avoidance method further includes:

[0120] S75. When the first target point information is landing point information, the landing module of the autonomous exploration framework modifies the second flight path based on the landing point information to obtain a third flight path.

[0121] refer to Figure 2 As shown, the autonomous exploration framework further includes a landing module. The landing module is used to modify the second flight path and perform a landing mission.

[0122] In an embodiment of the present application, the landing point information is a block with a helipad mark. The drone identifies the block through the target detection algorithm and performs the landing mission based on the block.

[0123] S76: Adjust the flight attitude of the UAV based on the third flight path to perform a landing mission.

[0124] In some embodiments, after the step S76, the drone obstacle avoidance method further includes:

[0125] S77: Acquire a second ground image.

[0126] The second ground image is collected by the downward-looking camera. The second ground image is an RGB image.

[0127] S78. Modify the third flight path based on the second ground image to obtain a fourth flight path.

[0128] Specifically, the step S78 includes:

[0129] S781, identifying the pixel center position of the landing point information at the second ground image;

[0130] S782, by calculating the relative position of the pixel center position and the center of the downward-looking camera in the coordinate system of the second ground image, and combining the focal length and pixel size of the downward-looking camera, the actual position of the pixel center position on the ground is deduced, and the distance information between the pixel center position and the center of the downward-looking camera in the x-axis and y-axis directions is obtained;

[0131] S783. Modify the third flight path based on the distance information to obtain the fourth flight path.

[0132] S79. Adjust the flight attitude of the UAV based on the fourth flight path.

[0133] In some embodiments, after step S5, the drone obstacle avoidance method further includes:

[0134] S81. Acquire a third ground image.

[0135] The third ground image is collected by the downward-looking camera. The third ground image contains two-dimensional code information. The two-dimensional code information is an Aruco two-dimensional code.

[0136] S82. Use the two-dimensional code recognition module of the autonomous exploration framework to analyze and process the third ground image to obtain second target point information.

[0137] refer to Figure 2 As shown, the autonomous exploration framework further includes the two-dimensional code recognition module. The two-dimensional code recognition module is mainly used to recognize the two-dimensional code information.

[0138] exist Figure 2 In the present invention, the input data of the autonomous exploration framework and each module of the autonomous exploration framework are divided into a data input layer, a positioning layer, a task point recognition layer and a motion planning layer, so as to understand the input and processing flow of the autonomous exploration framework.

[0139] S83: Feedback the second target point information to the client.

[0140] The client is a smart terminal used by a user, and the smart terminal includes but is not limited to a laptop computer, a tablet computer, and a mobile phone.

[0141] In actual applications, the QR code information can be placed along the path of the warehouse robot. When the downward-looking camera of the drone detects or collects the QR code information, since each QR code information has a unique ID and the placement position of each QR code information in the warehouse is known, the user can understand the precise position of the drone in the warehouse in real time.

[0142] <Drone obstacle avoidance device>

[0143] like Figure 3 FIG. 1 is a block diagram of a drone obstacle avoidance device provided in accordance with the second aspect of the present application. Figure 3 In the embodiment, the unmanned aerial vehicle obstacle avoidance device 100 comprises:

[0144] An acquisition module 101 is used to acquire first point cloud data;

[0145] A real-time positioning information output module 102 is used to input the first point cloud data into the positioning and mapping module of the autonomous exploration framework for data analysis, and output the real-time positioning information and the second point cloud data;

[0146] The point cloud information output module 103 to be processed is used to input the second point cloud data into the point cloud segmentation module of the autonomous exploration framework for segmentation processing, and output the point cloud information to be processed;

[0147] The task point information output module 104 is used to input the to-be-processed point cloud information into the recognition and penetration module of the autonomous exploration framework for structural feature matching processing, and output the task point information;

[0148] A first flight path output module 105, used for inputting the mission point information and the real-time positioning information into the local planning module of the autonomous exploration framework for local path planning, and outputting a first flight path for obstacle avoidance;

[0149] The flight mission execution module 106 is configured to execute the flight mission based on the first flight path.

[0150] <Drone>

[0151] like Figure 4 As shown, in the third aspect of an embodiment of the present application, a drone is provided, wherein the drone 200 includes: a processor 201 and a memory 202, wherein the memory 202 is used to store a computer program, and the processor 201 is used to call and run the computer program stored in the memory 202 to execute the steps of the drone obstacle avoidance method described in the first aspect of the embodiment of the present application.

[0152] The drone mentioned in the embodiment of the present application is a multi-rotor drone. In other embodiments, the specific structure of the drone can be set by those skilled in the art according to actual needs.

[0153] <Computer Readable Storage Medium>

[0154] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, and the computer program enables a computer to execute the steps of the drone obstacle avoidance method described in the first aspect of the embodiment of the present application.

[0155] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0156] The technical features of the above embodiments can be combined without changing the basic principles of the present invention. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of patent protection of the present application shall be subject to the attached claims.

Claims

1. A method for avoiding obstacles in a drone, characterized in that: include: Acquire the first point cloud data; Input the first point cloud data into the positioning and mapping module of the autonomous exploration framework for data analysis, and output real-time positioning information and second point cloud data; Inputting the second point cloud data into the point cloud segmentation module of the autonomous exploration framework for segmentation processing, and outputting the point cloud information to be processed; Input the point cloud information to be processed into the recognition and penetration module of the autonomous exploration framework for structural feature matching processing, and output the task point information; Inputting the mission point information and the real-time positioning information into a local planning module of the autonomous exploration framework for local path planning, and outputting a first flight path for obstacle avoidance; A flight mission is performed based on the first flight path.

2. The unmanned aerial vehicle obstacle avoidance method according to claim 1, characterized in that: The step of inputting the first point cloud data into the positioning and mapping module of the autonomous exploration framework for data analysis and outputting real-time positioning information and second point cloud data comprises: Performing denoising and dedistortion processing on the first point cloud data to obtain the second point cloud data; The first point cloud data is input into the extended Kalman filter of the positioning and mapping module, and the extended Kalman filter fuses the first point cloud data with the first IMU data to obtain the real-time positioning information.

3. The unmanned aerial vehicle obstacle avoidance method according to claim 1, characterized in that: The step of inputting the second point cloud data into the point cloud segmentation module of the autonomous exploration framework for segmentation processing and outputting the point cloud information to be processed comprises: The point cloud segmentation module calculates the normal Z value corresponding to each point in the second point cloud data by using a surface normal estimation method; The normal Z value corresponding to each point in the second point cloud data is threshold filtered to obtain non-ground point cloud information, and the non-ground point cloud information is input into the RANSAC algorithm of the point cloud segmentation module. The RANSAC algorithm analyzes the non-ground point cloud information, excludes the noise point cloud information in the non-ground point cloud information, fits the plane model of the ground, and segments the ground point cloud from the second point cloud data to obtain the point cloud information to be processed.

4. The unmanned aerial vehicle obstacle avoidance method according to claim 1, characterized in that: The step of inputting the to-be-processed point cloud information into the recognition and penetration module of the autonomous exploration framework for structural feature matching processing and outputting the task point information comprises: Selecting one or more points from the point cloud information to be processed as initial seed points; The points adjacent to each initial seed point are classified by combining the Euclidean clustering algorithm and the neighboring search algorithm to obtain the cluster corresponding to each initial seed point. The structural features of all the clusters are matched with preset structural features to obtain the task point information.

5. The unmanned aerial vehicle obstacle avoidance method according to claim 1, characterized in that: After the step of performing the flight mission based on the first flight path, the unmanned aerial vehicle obstacle avoidance method further includes: Acquire a first ground image; The target detection module of the autonomous exploration framework is used to analyze and process the first ground image to obtain first target point information; Modify the first flight path based on the first target point information to obtain a second flight path; The flight attitude of the UAV is adjusted based on the second flight path.

6. The unmanned aerial vehicle obstacle avoidance method according to claim 5, characterized in that: After the step of adjusting the flight attitude of the UAV based on the second flight path, the UAV obstacle avoidance method further includes: When the first target point information is landing point information, the landing module of the autonomous exploration framework modifies the second flight path based on the landing point information to obtain a third flight path; The flight attitude of the UAV is adjusted based on the third flight path to perform a landing mission.

7. The unmanned aerial vehicle obstacle avoidance method according to claim 6, characterized in that: After the step of adjusting the flight attitude of the UAV based on the third flight path and performing the landing task, the UAV obstacle avoidance method further includes: acquiring a second ground image; modifying the third flight path based on the second ground image to obtain a fourth flight path; The flight attitude of the UAV is adjusted based on the fourth flight path.

8. An obstacle avoidance device for a drone, characterized in that: include: An acquisition module, used for acquiring first point cloud data; A real-time positioning information output module, used to input the first point cloud data into the positioning and mapping module of the autonomous exploration framework for data analysis, and output real-time positioning information and second point cloud data; A point cloud information output module to be processed, used for inputting the second point cloud data into the point cloud segmentation module of the autonomous exploration framework for segmentation processing, and outputting the point cloud information to be processed; A task point information output module is used to input the to-be-processed point cloud information into the recognition and penetration module of the autonomous exploration framework for structural feature matching processing and output task point information; A first flight path output module, used for inputting the mission point information and the real-time positioning information into the local planning module of the autonomous exploration framework for local path planning, and outputting a first flight path for obstacle avoidance; A flight mission execution module is used to execute the flight mission based on the first flight path.

9. A drone, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the steps of the drone obstacle avoidance method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the steps of the drone obstacle avoidance method described in any one of claims 1 to 7.

Images