Unmanned aerial vehicle obstacle avoidance method based on millimeter wave radar

Abstract

The invention discloses an unmanned aerial vehicle obstacle avoidance method based on a millimeter wave radar, and relates to the technical field of unmanned aerial vehicle obstacle avoidance. The unmanned aerial vehicle obstacle avoidance method based on the millimeter-wave radar comprises the following specific steps that a user constructs an environment map with an unmanned aerial vehicle as the center based on millimeter-wave radar data; the motion state of the obstacle is judged through detection sound waves, and when the static coordinate (x1, y1) of the obstacle is obtained, or when the vector # imgabs0 # millimeter wave radar obtains the vector # imgabs1 # of the dynamic coordinate of the obstacle, the predicted motion trail of the unmanned aerial vehicle and the motion trail of the obstacle are generated. Dynamic coordinates of a dynamic obstacle are obtained through the millimeter wave radar, multiple pieces of position information and position coordinates of the obstacle are obtained through detection sound waves returned at two adjacent times, the path track of the dynamic obstacle is integrated on the basis, and whether the dynamic obstacle has a contact point with follow-up flight of the unmanned aerial vehicle or not is judged through the execution module and a user; the unmanned aerial vehicle based on the millimeter-wave radar can avoid dynamic obstacles.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle obstacle avoidance, and in particular to a method for unmanned aerial vehicle obstacle avoidance based on millimeter wave radar. Background Art

[0002] With the rapid development of agricultural automation technology, plant protection drones have been widely used in the field of crop spraying. The actual farmland operating environment of plant protection drones is very complex. Electric poles, trees, people, animals, terrain differences, etc. can all become obstacles during the flight of drones, especially the common electric poles in farmland, which increase the risk of drone bombing. In order to avoid obstacles, users are required to intentionally avoid obstacles in the map when planning routes, which can easily lead to map fragmentation, increase workload, and reduce the operating efficiency of drones. Especially when the location of obstacles is unknown, the route cannot be accurately and reliably planned, resulting in the inability of drones to operate.

[0003] In the prior art, the image-based autonomous obstacle avoidance method is to hover and avoid obstacles when the camera recognizes obstacles. This type of solution cannot completely overcome the impact of low-light flight at night on camera imaging, and can only reduce the flight speed, resulting in reduced nighttime operation efficiency. The radar-based autonomous obstacle avoidance method hovers after encountering an obstacle and cannot continue autonomous flight. It must rely on manual control to avoid obstacles, which greatly reduces operation efficiency.

[0004] For example, a UAV obstacle avoidance method based on millimeter-wave radar is disclosed in China Invention Publication No. CN108594849A. This method uses radar feedback to detect obstacles and establish obstacle avoidance routes, which effectively increases the flight speed, eliminates the need for manual control to avoid obstacles, improves operational efficiency, and improves the efficiency of night operations.

[0005] However, in the process of avoiding dynamic obstacles by the above method, since the coordinates of the dynamic obstacles in the grid map are constantly changing, when calculating the absolute coordinates of the obstacles, the distance information and angle information of the detection sound waves returned in the same time period are constantly changing, so the absolute coordinates are constantly changing. As a result, there are safety hazards when the above method is used to avoid dynamic obstacles. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a UAV obstacle avoidance method based on millimeter-wave radar, which solves the problems raised in the above background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for avoiding obstacles for unmanned aerial vehicles based on millimeter wave radar, comprising the following specific steps:

[0008] S1: Based on millimeter-wave radar data, users build an environmental map with the drone as the center. During the flight of the drone, the millimeter-wave radars located in front of the drone and on the left and right sides continuously send out detection sound waves. The receiving module transmits the returned detection sound waves to the processing module to preliminarily understand the distance between the obstacles encountered in the current environment and the drone, and cooperate with the visual sensor to preliminarily build an environmental map, making up for the shortcomings of millimeter-wave radar in environmental perception.

[0009] S2: Determine the motion state of the obstacle by detecting the sound waves returned at two adjacent times, and obtain the static coordinates (x1, y1) or dynamic coordinates of the obstacle in the environment map based on the millimeter wave radar data.

[0010] S3: When the millimeter-wave radar obtains the static coordinates (x1, y1) of the obstacle, it determines whether the static coordinates (x1, y1) are on the current flight route based on the drone's position, speed, speed direction and altitude information provided by the millimeter-wave radar and the drone's coordinates (x0, y0) at this time; if so, the drone slows down, and the convolutional neural network architecture generates an obstacle avoidance flight route that does not pass through the static coordinates (x1, y1) based on the original flight route.

[0011] S4: The millimeter wave radar obtains the vector of the dynamic coordinates of the obstacle When the UAV speed and speed direction provided by the millimeter wave radar are combined with the vector The direction and size of the obstacle are used to generate the expected motion trajectory P1 of the drone and the motion trajectory P2 of the obstacle. If P1 and P2 have more than one intersection, the drone slows down, and the convolutional neural network architecture generates an obstacle avoidance flight route P3 that does not pass through the motion trajectory P2 based on the original flight route.

[0012] A further improvement of the technical solution of the present invention is that the environmental map in S1 also includes a digital signal transmitted by a visual sensor in addition to the millimeter-wave radar data. The detection image of the camera in the visual sensor can be directly presented to the user end, and the detection image presented to the user end is converted from low resolution to high resolution by the high-resolution imaging unit of the millimeter-wave radar.

[0013] A further improvement of the technical solution of the present invention is that the vector of the dynamic coordinates in S2 A set of parallel narrow-band filters in the millimeter-wave radar covers the expected Doppler frequency range. By detecting the filter output, the speed of the obstacle, i.e., the vector The size of the obstacle is determined by the direction of movement of the obstacle, i.e., the vector, based on two or more detection sound waves returned at adjacent times. direction, where when there is relative motion between the target and the millimeter-wave radar, the frequency of the received echo signal will change. This phenomenon is called the Doppler effect. The Doppler frequency is proportional to the radial velocity of the target. Therefore, the speed of the target can be estimated by detecting the Doppler frequency, and the direction of movement of the obstacle can be determined by two or more detection sound waves returned at adjacent times, providing a basis for subsequent UAVs to avoid dynamic obstacles.

[0014] A further improvement of the technical solution of the present invention is that the obstacle avoidance flight route in S3 is generated by an execution module including a convolutional neural network architecture, and the execution module includes a computing unit, a learning unit and a database. The straight-line distance L1 between the drone's own coordinates (x0, y0) and the stationary coordinates (x1, y1) is calculated by the computing unit, and based on the drone speed V1 provided by the millimeter-wave radar, the possible contact time t1 is obtained. If the stationary coordinates (x1, y1) are on the current flight route, the obstacle avoidance flight route that does not pass through the stationary coordinates (x1, y1) is regenerated within t1.

[0015] A further improvement of the technical solution of the present invention is that: the learning unit completes the inductive storage of data when the drone encounters obstacles during its flight route, and stores the processed data in a database. After the drone encounters a second obstacle with the same static coordinates in the environment map constructed with itself as the center, the execution module generates an obstacle avoidance decision based on the processed data in the database. The learning unit is based on a machine learning algorithm to train the drone to identify and classify obstacles, improve obstacle avoidance efficiency, and support the drone's autonomous decision-making and planning for fixed obstacles.

[0016] A further improvement of the technical solution of the present invention is that the generation of the obstacle motion trajectory P2 in S4 includes the following specific steps:

[0017] Q1: Multiple location information and coordinates of obstacles are obtained from two or more detection sound waves returned at adjacent times;

[0018] Q2: Integrate the multiple location coordinates obtained in Q1 into an environment map built around the drone;

[0019] Q3: The multiple position coordinates in Q2 are integrated and calculated by the A-Star algorithm to obtain the obstacle path trajectories of multiple position coordinates;

[0020] Q4: The position information and position coordinates obtained from the subsequent detection sound waves returned at multiple adjacent times are used as sampling information to verify the obstacle path trajectory. If the verification passes, the subsequent position information and position coordinates are integrated into the original obstacle path trajectory to obtain the obstacle's motion trajectory P2. If the verification fails, return to Q2 to reintegrate the multiple coordinates into the environment map built with the drone as the center.

[0021] A further improvement of the technical solution of the present invention is that: multiple position detection images of obstacles obtained by the camera in the visual sensor in Q1 are presented to the user end to assist in verifying the obstacle path trajectory in Q3.

[0022] A further improvement of the technical solution of the present invention is that if there is no intersection between the obstacle's motion trajectory P2 and the drone's predicted motion trajectory P1, the shortest distance between the obstacle's motion trajectory P2 and the drone's predicted motion trajectory P1 must be less than the drone's width.

[0023] Compared with the prior art, the invention has the following beneficial effects: the dynamic coordinates of the dynamic obstacle are obtained by the millimeter wave radar, and multiple position information and position coordinates of the obstacle are obtained by the detection sound waves returned at two or more adjacent times. On this basis, the path trajectory of the dynamic obstacle is integrated, and the execution module and the user determine whether the dynamic obstacle has a contact point with the subsequent flight of the drone, so as to realize the drone avoiding dynamic obstacles based on the millimeter wave radar;

[0024] The learning unit is used to train drones to identify and classify obstacles based on machine learning algorithms, improve obstacle avoidance efficiency, and support autonomous decision-making and planning of drones for fixed obstacles, avoiding frequent manual autonomous control of drones by users, and improving drone flight efficiency.

[0025] The millimeter-wave radars located in front of the drone and on both sides continuously emit detection sound waves. The receiving module transmits the returned detection sound waves to the processing module, which can initially understand the distance between the obstacles encountered in the current environment and the drone, and cooperate with the visual sensor to preliminarily build an environmental map, thus making up for the shortcomings of millimeter-wave radar in environmental perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of the UAV obstacle avoidance method in the UAV obstacle avoidance method based on millimeter wave radar;

[0027] Figure 2 A flowchart for constructing dynamic trajectories of obstacles in the obstacle avoidance method of UAV based on millimeter wave radar;

[0028] Figure 3 A schematic diagram of changing the obstacle avoidance flight route when the UAV encounters a stationary obstacle in the UAV obstacle avoidance method based on millimeter wave radar;

[0029] Figure 4 A schematic diagram of changing the obstacle avoidance flight route when the UAV encounters a dynamic obstacle in the UAV obstacle avoidance method based on millimeter wave radar;

[0030] Figure 5This is a schematic diagram of a UAV encountering a dynamic obstacle but the flight path does not touch the motion trajectory in a UAV obstacle avoidance method based on millimeter-wave radar. DETAILED DESCRIPTION

[0031] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0032] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0033] In addition, in order to better illustrate the present application, numerous specific details are provided in the specific embodiments below. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, and elements well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0034] Reference Figure 1-Figure 5 , the present invention provides three technical solutions:

[0035] Embodiment 1:

[0036] A method for avoiding obstacles of a UAV based on millimeter wave radar includes the following specific steps:

[0037] S1: Based on millimeter-wave radar data, users build an environmental map with the drone as the center. During the flight of the drone, the millimeter-wave radars located in front of the drone and on the left and right sides continuously send out detection sound waves. The receiving module transmits the returned detection sound waves to the processing module to preliminarily understand the distance between the obstacles encountered in the current environment and the drone, and cooperate with the visual sensor to preliminarily build an environmental map, making up for the shortcomings of millimeter-wave radar in environmental perception.

[0038] S2: Determine the motion state of the obstacle by detecting the sound waves returned at two adjacent times, and obtain the static coordinates (x1, y1) or dynamic coordinates of the obstacle in the environment map based on the millimeter wave radar data.

[0039] S3: When the millimeter-wave radar obtains the static coordinates (x1, y1) of the obstacle, it determines whether the static coordinates (x1, y1) are on the current flight route based on the drone's position, speed, speed direction and altitude information provided by the millimeter-wave radar and the drone's coordinates (x0, y0) at this time; if so, the drone slows down, and the convolutional neural network architecture generates an obstacle avoidance flight route that does not pass through the static coordinates (x1, y1) based on the original flight route.

[0040] S4: The millimeter wave radar obtains the vector of the dynamic coordinates of the obstacle When the UAV speed and speed direction provided by the millimeter wave radar are combined with the vector The direction and size of the obstacle are used to generate the expected motion trajectory P1 of the drone and the motion trajectory P2 of the obstacle. If P1 and P2 have more than one intersection, the drone slows down, and the convolutional neural network architecture generates an obstacle avoidance flight route P3 that does not pass through the motion trajectory P2 based on the original flight route.

[0041] The environmental map in S1 includes digital signals transmitted by visual sensors in addition to millimeter-wave radar data. The detection image of the camera in the visual sensor can be directly presented to the user end, and the detection image presented to the user end is converted from low resolution to high resolution by the high-resolution imaging unit of the millimeter-wave radar.

[0042] Embodiment 2:

[0043] Based on the first embodiment: the vector of the dynamic coordinates in S2 A set of parallel narrow-band filters in the millimeter-wave radar covers the expected Doppler frequency range. By detecting the filter output, the speed of the obstacle, i.e., the vector The size of the obstacle is determined by the direction of movement of the obstacle, i.e., the vector, based on two or more detection sound waves returned at adjacent times. When there is relative motion between the target and the millimeter-wave radar, the frequency of the received echo signal will change. This phenomenon is called the Doppler effect. The Doppler frequency is proportional to the radial velocity of the target. Therefore, the speed of the target can be estimated by detecting the Doppler frequency, and the direction of movement of the obstacle can be determined by two or more detection sound waves returned at adjacent times, which provides a basis for the subsequent UAV to avoid dynamic obstacles.

[0044] The obstacle avoidance flight route in S3 is generated by an execution module including a convolutional neural network architecture, and the execution module includes a computing unit, a learning unit and a database. The computing unit calculates the straight-line distance L1 between the drone's own coordinates (x0, y0) and the stationary coordinates (x1, y1), and based on the drone speed V1 provided by the millimeter-wave radar, the possible contact time t1 is obtained. If the stationary coordinates (x1, y1) are on the current flight route, the obstacle avoidance flight route that does not pass through the stationary coordinates (x1, y1) is regenerated within t1.

[0045] The calculation formula of L1 is: but Furthermore, the slope between the stationary coordinates (x1, y1) and the drone's own coordinates (x0, y0) is When the execution module calculates whether the drone touches an obstacle within t1, the flight direction of the drone should also be considered. Therefore, based on the result obtained by the slope k, the angle between the flight direction of the drone and the x-axis is θ. When tanθ is greater than k, the drone does not touch the obstacle. When tanθ is less than or equal to k, an obstacle avoidance flight route with stationary coordinates (x1, y1) needs to be generated.

[0046] The learning unit completes the inductive storage of the data when the drone encounters obstacles during its flight route, and stores the processed data in a database. When the drone encounters a second obstacle with the same static coordinates in the environment map built with itself as the center, the execution module generates obstacle avoidance decisions based on the processed data in the database. The learning unit is based on a machine learning algorithm to train the drone to identify and classify obstacles, improve obstacle avoidance efficiency, and support the drone's autonomous decision-making and planning for fixed obstacles.

[0047] Embodiment three:

[0048] On the basis of the first and second embodiments: the generation of the obstacle motion trajectory P2 in S4 includes the following specific steps:

[0049] Q1: Multiple location information and coordinates of obstacles are obtained from two or more detection sound waves returned at adjacent times;

[0050] Q2: Integrate the multiple location coordinates obtained in Q1 into an environment map built around the drone;

[0051] Q3: The multiple position coordinates in Q2 are integrated and calculated by the A-Star algorithm to obtain the obstacle path trajectories of multiple position coordinates;

[0052] Q4: The position information and position coordinates obtained from the subsequent detection sound waves returned at multiple adjacent times are used as sampling information to verify the obstacle path trajectory. If the verification passes, the subsequent position information and position coordinates are integrated into the original obstacle path trajectory to obtain the obstacle's motion trajectory P2. If the verification fails, return to Q2 to reintegrate the multiple coordinates into the environment map built with the drone as the center.

[0053] The multiple position detection images of the obstacle obtained by the camera in the visual sensor in Q1 are presented to the user end to assist in verifying the obstacle path trajectory in Q3.

[0054] If there is no intersection between the obstacle's motion trajectory P2 and the drone's predicted motion trajectory P1, the shortest distance between the obstacle's motion trajectory P2 and the drone's predicted motion trajectory P1 must be less than the drone's width.

[0055] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0056] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for avoiding obstacles for unmanned aerial vehicles based on millimeter wave radar, characterized in that: The specific steps include: S1: Users build an environment map centered on the drone based on millimeter-wave radar data; S2: Determine the motion state of the obstacle by the detection sound waves returned at two adjacent times, and obtain the static coordinates (x1, y1) or dynamic coordinates of the obstacle in the environment map based on the millimeter wave radar data. S3: When the millimeter-wave radar obtains the static coordinates (x1, y1) of the obstacle, it determines whether the static coordinates (x1, y1) are on the current flight route based on the drone's position, speed, speed direction and altitude information provided by the millimeter-wave radar and the drone's coordinates (x0, y0) at this time; if so, the drone decelerates and generates an obstacle avoidance flight route that does not pass through the static coordinates (x1, y1) based on the original flight route; S4: The millimeter wave radar obtains the vector of the dynamic coordinates of the obstacle When the UAV speed and speed direction provided by the millimeter wave radar are combined with the vector The direction and size of the obstacle are used to generate the expected motion trajectory P1 of the drone and the motion trajectory P2 of the obstacle. If P1 and P2 have more than one intersection, the drone slows down and generates an obstacle avoidance flight trajectory P3 that does not pass through the motion trajectory P2 based on the original flight route.

2. The method for avoiding obstacles of unmanned aerial vehicles based on millimeter wave radar according to claim 1, characterized in that: The environmental map in S1 includes digital signals transmitted by visual sensors in addition to millimeter-wave radar data. The detection image of the camera in the visual sensor can be directly presented to the user end, and the detection image presented to the user end is converted from low resolution to high resolution by the high-resolution imaging unit of the millimeter-wave radar.

3. The method for avoiding obstacles for unmanned aerial vehicles based on millimeter wave radar according to claim 1, characterized in that: The vector of dynamic coordinates in S2 A set of parallel narrow-band filters in the millimeter-wave radar covers the expected Doppler frequency range. By detecting the filter output, the speed of the obstacle, i.e., the vector The size of the obstacle is determined by the direction of movement of the obstacle, i.e., the vector, based on two or more detection sound waves returned at adjacent times. direction.

4. The method for avoiding obstacles for unmanned aerial vehicles based on millimeter wave radar according to claim 1, characterized in that: The obstacle avoidance flight route in S3 is generated by an execution module including a convolutional neural network architecture, and the execution module includes a computing unit, a learning unit and a database. The computing unit calculates the straight-line distance L1 between the drone's own coordinates (x0, y0) and the stationary coordinates (x1, y1), and based on the drone speed V1 provided by the millimeter-wave radar, the possible contact time t1 is obtained. If the stationary coordinates (x1, y1) are on the current flight route, the obstacle avoidance flight route that does not pass through the stationary coordinates (x1, y1) is regenerated within t1.

5. The method for avoiding obstacles for unmanned aerial vehicles based on millimeter wave radar according to claim 4, characterized in that: The learning unit completes the inductive storage of data when the drone encounters obstacles during its flight route, and stores the processed data in a database. When the drone encounters a second obstacle with the same static coordinates in the environment map built with itself as the center, the execution module generates an obstacle avoidance decision based on the processed data in the database.

6. The method for avoiding obstacles of a UAV based on millimeter wave radar according to claim 1, characterized in that: The generation of the obstacle motion trajectory P2 in S4 includes the following specific steps: Q1: Multiple location information and coordinates of obstacles are obtained from two or more detection sound waves returned at adjacent times; Q2: Integrate the multiple location coordinates obtained in Q1 into an environment map built around the drone; Q3: The multiple position coordinates in Q2 are integrated and calculated by the A-Star algorithm to obtain the obstacle path trajectories of multiple position coordinates; Q4: The position information and position coordinates obtained from the subsequent detection sound waves returned at multiple adjacent times are used as sampling information to verify the obstacle path trajectory. If the verification passes, the subsequent position information and position coordinates are integrated into the original obstacle path trajectory to obtain the obstacle's motion trajectory P2. If the verification fails, return to Q2 to reintegrate the multiple coordinates into the environment map built with the drone as the center.

7. The method for avoiding obstacles for unmanned aerial vehicles based on millimeter wave radar according to claim 6, characterized in that: The multiple position detection images of the obstacle obtained by the camera in the visual sensor in Q1 are presented to the user end to assist in verifying the obstacle path trajectory in Q3.

8. The method for avoiding obstacles for unmanned aerial vehicles based on millimeter wave radar according to claim 1, characterized in that: If there is no intersection between the obstacle's motion trajectory P2 and the drone's predicted motion trajectory P1, the shortest distance between the obstacle's motion trajectory P2 and the drone's predicted motion trajectory P1 must be less than the drone's width.

Images