Unmanned aerial vehicle movement control method, device, equipment and storage medium

Abstract

Embodiments of the present application disclose a kind of unmanned plane's mobile control method, device, equipment and storage medium, belong to unmanned plane technical field, and laser radar is carried on unmanned plane, which comprises: calling laser radar is scanned along the moving direction of unmanned plane, obtains first radar data;In the case where it is determined that there is an obstacle in the moving direction based on first radar data, the type of the obstacle is determined based on first radar data;In the case where it is determined that the obstacle is a stationary obstacle, the edge profile of the stationary obstacle is determined based on first radar data;Based on edge profile, unmanned plane avoids stationary obstacle.This application is based on the characteristics of stationary obstacle, the edge profile of stationary obstacle is determined by first radar data, and unmanned plane is controlled to avoid stationary obstacle based on edge profile, without constructing real-time map for obstacle avoidance, the amount of calculation required for the obstacle avoidance control of unmanned plane can be reduced, and the moving efficiency of unmanned plane is improved.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a method, apparatus, device, and storage medium for controlling the movement of a UAV. Background Technology

[0002] Unmanned aerial vehicles (UAVs), or drones, are unmanned machines controlled by radio remote control equipment and their own programmed control devices. Currently, with scientific advancements and social progress, the application of UAVs is gradually increasing.

[0003] In related technologies, if an obstacle is detected while the drone is moving, it uses its onboard sensors (such as visual cameras, lidar, etc.) to perceive the surrounding environment in real time and build a map to determine the location of the obstacle on the map. Then, based on the real-time map, the drone is controlled to avoid the obstacle and continue moving.

[0004] However, in the aforementioned related technologies, it is necessary to build a map in real time for obstacle avoidance when an obstacle is detected, which involves a large amount of computation. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for controlling the movement of a drone, which can reduce the computational load required for obstacle avoidance control of the drone and improve the drone's movement efficiency. The technical solution is as follows: On one hand, embodiments of this application provide a method for motion control and detection of a drone, wherein the drone is equipped with a lidar, and the method includes: The lidar is invoked to scan along the movement direction of the UAV to obtain first radar data; If an obstacle is determined to exist in the direction of movement based on the first radar data, the type of the obstacle is determined based on the first radar data; If the obstacle is determined to be a stationary obstacle, the edge contour of the stationary obstacle is determined based on the first radar data; The drone is controlled to avoid stationary obstacles based on the edge contour.

[0006] On the other hand, embodiments of this application provide a mobile control device for a drone, wherein the drone is equipped with a lidar, and the mobile control device is used to implement the aforementioned mobile control method for the drone. The device includes: The radar scanning module is used to call the lidar to scan along the movement direction of the UAV and obtain radar data; A type determination module is used to determine the type of obstacle based on the radar data when it is determined that there is an obstacle in the direction of movement based on the radar data. A contour detection module is used to determine the edge contour of a stationary obstacle based on radar data when the obstacle is determined to be a stationary obstacle. A stationary avoidance module is used to control the UAV to avoid stationary obstacles based on the edge contour.

[0007] In another aspect, embodiments of this application provide a computer device, which includes a processor and a memory. The memory stores a computer program, which is loaded and executed by the processor to implement the aforementioned mobile control method for the unmanned aerial vehicle.

[0008] In another aspect, embodiments of this application provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned mobile control method for the unmanned aerial vehicle.

[0009] In another aspect, embodiments of this application provide a computer program product that, when run, causes a computer device to execute the aforementioned drone motion control method.

[0010] Compared with the prior art, the technical solution provided in this application can bring the following beneficial effects: By scanning the drone's movement direction with lidar and distinguishing between stationary and stationary obstacles, if a stationary obstacle is detected, the edge contour of the stationary obstacle is determined based on the characteristic of the obstacle being stationary. The drone is then controlled to avoid the stationary obstacle based on the edge contour. This eliminates the need to build a real-time map for obstacle avoidance, effectively reducing the amount of computation required for obstacle avoidance control of the drone and thus improving the drone's movement efficiency. Attached Figure Description

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a schematic diagram of a flight control system provided in one embodiment of this application; Figure 2 This is a flowchart of a drone motion control method provided in one embodiment of this application; Figure 3 This is a flowchart of a drone motion control method provided in another embodiment of this application; Figure 4 An exemplary schematic diagram of an obstacle avoidance method for moving obstacles is shown; Figure 5 An exemplary diagram illustrates another obstacle avoidance method for moving obstacles; Figure 6An exemplary diagram illustrates yet another obstacle avoidance method for moving obstacles; Figure 7 This is a flowchart of a drone motion control method provided in another embodiment of this application; Figure 8 This is a block diagram of a mobile control device for an unmanned aerial vehicle provided in one embodiment of this application; Figure 9 This is a block diagram of a mobile control device for a drone provided in another embodiment of this application. Detailed Implementation

[0012] The present invention will be further described in detail below with reference to specific embodiments: Please refer to Figure 1 The diagram illustrates a flight control system according to an embodiment of this application. The flight control system may include a drone 10 and a computer device 20.

[0013] The drone 10 is an unmanned machine. Exemplarily, the drone 10 can be a fixed-wing drone, a rotary-wing drone, an unmanned airship, a paragliding drone, or a flapping-wing drone, etc., and this application embodiment does not limit it to this type.

[0014] Computer device 20 is used to control the flight of drone 10. Exemplarily, computer device 20 can be an electronic device such as a mobile phone, tablet computer, wearable device, backend server, onboard computer of drone, server cluster, or PC (Personal Computer), and this application embodiment does not limit it to this.

[0015] Optionally, the drone 10 is equipped with a lidar. For example, with the direction of movement of the drone 10 as the front, the lidar is positioned directly above the drone 10. In this embodiment, during the movement of the drone 10, the computer device 20 invokes the lidar to scan along the drone's direction of movement, obtaining first radar data. Based on the first radar data, obstacle detection is performed to determine the type of obstacle. If the obstacle is determined to be a stationary obstacle, the edge contour of the stationary obstacle is determined based on the first radar data, and the drone 10 is controlled to avoid the stationary obstacle based on this edge contour. For example, the lidar can be a ToF (Time of Flight) lidar, a triangulation lidar, a mechanical lidar, a hybrid solid-state lidar, or an all-solid-state lidar, etc., and this embodiment does not limit the specific type.

[0016] Optionally, the drone 10 is also equipped with an image acquisition device with depth information acquisition capabilities. For example, the image acquisition device is positioned near the aforementioned lidar. In this embodiment, when the computer device 20 determines that an obstacle is a moving obstacle based on the first radar data, it invokes the image acquisition device and the aforementioned lidar to acquire at least two sets of dynamic data. Then, based on these two sets of dynamic data, it determines the speed and direction of movement of the moving obstacle, and controls the drone to avoid the moving obstacle based on this speed and direction. For example, the image acquisition device can be any device with depth information acquisition capabilities, such as a depth camera or a binocular camera; this embodiment does not limit its capabilities.

[0017] Optionally, the drone 10 is equipped with an image acquisition device for detecting ground information. Exemplarily, the image acquisition device is positioned on the side of the drone 10 closest to the ground when it is moving. In this embodiment, during the movement of the drone 10, the computer device 20 controls the drone 10 to land in a pre-set landing area when a landing marker is detected by the image acquisition device. Exemplarily, the image acquisition device can be any device with image acquisition capabilities, such as a camera or camcorder; this embodiment does not limit its use.

[0018] Optionally, the image acquisition device and the image capture device described above may be the same device or different devices, and this application embodiment does not limit this.

[0019] Optionally, in the flight control system, the number of UAVs 10 controlled by the computer device 20 can be any value. The number of UAVs 10 can be flexibly set and adjusted according to the actual situation. This application embodiment does not limit this.

[0020] Optionally, the drone 10 and the computer device 20 communicate via a network.

[0021] Please refer to Figure 2 This document illustrates a flowchart of a motion control method for a drone provided in one embodiment of this application. The method is applied to... Figure 1 The computer device 20 in the flight control system shown. The method may include the following steps (201-204): Step 201: Call the lidar to scan along the direction of the UAV's movement to obtain the first radar data.

[0022] LiDAR is used for real-time obstacle detection during the movement of a drone. In this embodiment, during the movement of the drone, a computer device invokes the LiDAR to scan along the direction of the drone's movement to obtain first radar data.

[0023] Optionally, during the movement of the drone, the LiDAR scans the drone in real time to obtain first radar data. This first radar data includes at least one set of data, and different data sets correspond to different timestamps. For example, the LiDAR uses a first time interval as the scanning detection method, scanning along the drone's movement direction at regular intervals to obtain a set of data, and the timestamp corresponding to this set of data is the current moment. The first time interval can be any value, and operators can flexibly set and adjust it according to actual conditions, such as 0.1 seconds, 0.5 seconds, 1 second, or 5 seconds, etc. This embodiment does not limit this.

[0024] Optionally, each set of the first radar data includes the distance between at least one laser spot and the UAV. For example, when the lidar is scanning, it emits more than one laser beam along the direction of the UAV's movement. When the laser beam contacts an object, it forms a laser spot. The distance between the formed laser spot and the UAV is recorded, along with the emission angle of the laser beam corresponding to that spot, thus obtaining a set of data.

[0025] Alternatively, since the drone can fly in the air, the detection range of the lidar is a cone-shaped three-dimensional space with the lidar as the apex.

[0026] Step 202: If it is determined that there is an obstacle in the direction of movement based on the first radar data, determine the type of obstacle based on the first radar data.

[0027] In this embodiment of the application, after acquiring the first radar data, the computer device determines the type of obstacle based on the first radar data if it determines that there is an obstacle in the direction of movement.

[0028] Optionally, since the first radar data is generated only when the laser comes into contact with an object, the computer device, when acquiring the first radar data, determines that there is an obstacle in the direction of movement of the UAV based on the first radar data.

[0029] Optionally, the obstacle types include stationary obstacles and moving obstacles. Stationary obstacles refer to obstacles whose attitude and position do not change, such as tall buildings and mountains; moving obstacles refer to obstacles whose attitude and / or position change, such as a single bird or flock of birds in flight, or tree branches swaying in the wind. Since the first radar data includes more than one data set, optionally, the data set with the earliest timestamp in the first radar data is the initial data set, and other data sets are subsequent data sets. For example, for the initial data set, the computer device determines the initial distribution coordinates of the light spot in the three-dimensional coordinate system based on the distance between the light spot and the UAV recorded in the initial data set and the emission angle of the laser corresponding to the light spot. Then, based on the distance between the light spot and the UAV recorded in the subsequent data sets and the emission angle of the laser corresponding to the light spot, the subsequent distribution coordinates of the light spot in the three-dimensional coordinate system are determined. Finally, based on the initial and subsequent distribution coordinates, combined with the positional changes of the UAV, the type of obstacle is determined. The three-dimensional coordinate system refers to the three-dimensional coordinate system established with the aforementioned lidar as the origin, the direction of movement as the X-axis, and the direction perpendicular to the ground (standard horizontal plane) upward as the Z-axis.

[0030] For example, after determining the initial and subsequent distribution coordinates, the computer device uses the timestamps corresponding to the initial and subsequent data sets as a reference to determine at least one time period. Using the time period as the basic unit, it acquires the positional changes of the UAV within that time period. Then, based on the positional changes of the UAV, it determines the transformation matrix of the three-dimensional coordinate system. Based on this transformation matrix, it processes the initial distribution coordinates to obtain the predicted distribution coordinates. Subsequently, if the predicted distribution coordinates are the same as the subsequent distribution coordinates, it is determined that the obstacle's attitude and position have not changed during that time period, i.e., the obstacle is a stationary obstacle. Conversely, if the predicted distribution coordinates are different from the subsequent distribution coordinates, it is determined that the obstacle's attitude and / or position have changed during that time period, i.e., the obstacle is a moving obstacle.

[0031] It should be noted that in the embodiments of this application, it is considered that some obstacles may change from static to dynamic due to environmental influences (e.g., trees are static obstacles in light winds, while trees move with the wind in strong winds and are dynamic obstacles). In order to distinguish the type of obstacle based solely on the first radar data without acquiring environmental data, the first radar data includes more than one set of data. Based on the light spot distribution coordinates corresponding to multiple timestamps, the type of obstacle can be determined more accurately.

[0032] Optionally, the number of obstacles detected by the first radar data may be one or more, and this embodiment of the application does not limit this. Exemplarily, after determining the initial distribution coordinates, the computer device uses the coordinate distance between the light spots as a reference and a clustering algorithm to determine that the high-density distributed light spots are light spots caused by the same obstacle, thereby determining the number of obstacles corresponding to the initial data set; furthermore, after obtaining the above-mentioned change matrix, the type of each obstacle is determined with the obstacle as the basic unit.

[0033] It should be noted that, in this embodiment of the application, considering that the birds in the flock are moving in roughly the same direction, in order to reduce the computational pressure caused by obstacle classification, the flock is directly regarded as a complete moving obstacle by clustering. Subsequently, the drone is controlled to avoid obstacles by treating the flock as a whole. This reduces the computational pressure and effectively improves the success rate of obstacle avoidance based on moving obstacles.

[0034] Step 203: If the obstacle is determined to be a stationary obstacle, the edge contour of the stationary obstacle is determined based on the first radar data.

[0035] In this embodiment of the application, when the obstacle is determined to be a stationary obstacle, the computer device determines the edge contour of the stationary obstacle based on the first radar data.

[0036] Optionally, the computer device determines the distribution coordinates of the light spot in the above three-dimensional coordinate system based on the distance between the light spot recorded by the first radar data and the UAV and the emission angle of the laser corresponding to the light spot, obtains the point cloud data corresponding to the first radar data, and then performs edge contour detection on the point cloud data to obtain the edge contour of the stationary obstacle.

[0037] Step 204: Control the drone to avoid stationary obstacles based on edge contours.

[0038] In this embodiment, after obtaining the aforementioned edge contour, the computer device controls the drone to avoid stationary obstacles based on the edge contour. Optionally, the computer device controls the drone to bypass the edge contour, so that the drone avoids the stationary obstacle.

[0039] In summary, the technical solution provided in this application uses LiDAR to scan the movement direction of the UAV and distinguish the type of obstacle by whether it is stationary. When it is determined that the detected obstacle is a stationary obstacle, the edge contour of the stationary obstacle is determined by the first radar data based on the characteristic that the obstacle is stationary. Then, the UAV is controlled to avoid the stationary obstacle based on the edge contour. There is no need to build a real-time map for obstacle avoidance, which can effectively reduce the amount of computation required for obstacle avoidance control of the UAV, thereby improving the movement efficiency of the UAV from the side.

[0040] The following section introduces obstacle avoidance methods for stationary obstacles.

[0041] In an exemplary embodiment, step 203 above includes the following steps: 1. Based on the location of the UAV, obtain at least one historical edge contour detected by the UAV at that location.

[0042] In this embodiment of the application, when controlling the drone to avoid the aforementioned stationary obstacle, the computer device acquires at least one historical edge contour detected by the drone at the location of the drone.

[0043] The drone's location is used to indicate its latitude, longitude, and altitude. Optionally, computer equipment determines the drone's latitude and longitude using a positioning system and determines its altitude using an onboard altitude detection device. For example, the positioning system includes the BeiDou Navigation Satellite System, the Global Positioning System (GPS), etc.; the altitude detection device includes a barometer, an ultrasonic sensor, a binocular camera, etc.

[0044] Historical drones refer to drones that passed through the aforementioned locations before this drone. Optionally, after obtaining the location of the drone, the computer device uses that location as an index to query the historical database to determine at least one historical drone that passed through that location, and then obtains the historical edge contour detected by that at least one historical drone at that location from the historical database.

[0045] 2. Match the edge contour with the historical edge contour to obtain the contour similarity corresponding to each historical edge contour.

[0046] In this embodiment of the application, after obtaining the above-mentioned edge contour, the computer device matches the edge contour with the historical edge contour to obtain the contour similarity corresponding to each historical edge contour.

[0047] Optionally, the computer device places the edge contour and the historical edge contour in the same three-dimensional coordinate system, performs translation and rotation operations on the historical edge contour to control its overlap with the edge contour, and determines the contour similarity corresponding to the historical edge contour based on the degree of overlap between the two edge contours. There is a positive correlation between the degree of overlap and the contour similarity; that is, the higher the degree of overlap, the greater the contour similarity, and vice versa. For example, the three-dimensional coordinate system is the aforementioned three-dimensional coordinate system. It should be noted that, since the obstacle has specific dimensions, the operations performed by the computer device when controlling the overlap between the historical edge contour and the edge contour do not include scaling operations.

[0048] 3. Among at least one historical UAV with a contour similarity greater than the similarity threshold, the historical UAV with the highest contour similarity is identified as the reference target.

[0049] In this embodiment of the application, after obtaining the contour similarity corresponding to each of the above-mentioned historical edge contours, the computer device determines the historical drone with the largest contour similarity as the reference target among at least one historical drone with a contour similarity greater than the similarity threshold.

[0050] Optionally, the similarity threshold can be any value. Staff can flexibly set and adjust the similarity threshold according to the actual situation, such as 100%, 99.9%, 95%, etc. This application embodiment does not limit this.

[0051] 4. Determine the historical obstacle avoidance paths of the reference target at different locations for the obstacle avoidance paths of the UAV for stationary obstacles.

[0052] In this embodiment of the application, after determining the aforementioned reference target, the computer device determines the historical obstacle avoidance path of the reference target at the aforementioned location for the reference obstacle as the obstacle avoidance path of the UAV for the aforementioned stationary obstacle.

[0053] The reference obstacle refers to the obstacle corresponding to the historical edge contour with the highest contour similarity. Optionally, the computer device obtains the historical obstacle avoidance path made by the reference target for the historical edge contour with the highest contour similarity, and then determines the historical obstacle avoidance path as the obstacle avoidance path of the UAV for stationary obstacles.

[0054] 5. Control the UAV to avoid stationary obstacles based on obstacle avoidance path.

[0055] In this embodiment of the application, after obtaining the obstacle avoidance path, the computer device controls the drone to avoid stationary obstacles based on the obstacle avoidance path.

[0056] It should be noted that the above description applies to the case where there are historical drones with contour similarity greater than the similarity threshold. In the absence of historical drones with contour similarity greater than the above similarity threshold, the computer equipment directly performs real-time obstacle avoidance for stationary obstacles based on the above edge contours.

[0057] Optionally, in the absence of any historical drones with a contour similarity greater than the aforementioned similarity threshold, the computer device acquires the distance between each edge point in the edge contour and the drone. Here, an edge point refers to the smallest constituent unit of the edge contour. Optionally, in this embodiment, the size of the edge point is the diameter of the laser emitted by the lidar, i.e., one edge point corresponds to one laser spot. The computer device determines the distance between each edge point and the drone based on the distance recorded by the aforementioned first radar data.

[0058] Optionally, after obtaining the distance between each edge point in the edge profile and the drone, the computer device determines the nearest edge point as the target point.

[0059] Optionally, after determining the target point, the computer device controls the drone to bypass the target point and avoid stationary obstacles. For example, during movement, the computer device invokes the aforementioned lidar to detect the distance between the drone and the target point in real time. Furthermore, if the distance between the drone and the target point is equal to a distance threshold, the computer device controls the drone to bypass the target point and move from the obstacle-prone area to the obstacle-free area.

[0060] Optionally, after controlling the drone to bypass the target point and avoid stationary obstacles, the computer device acquires the drone's movement path. The movement path refers to the route the drone takes from the aforementioned location to avoid the stationary obstacles.

[0061] Optionally, after acquiring the movement path, the computer device records the correspondence between the movement path, the location, and the edge contour. For example, after recording, the drone is a historical drone that has passed through the aforementioned location, the edge contour is the historical edge contour detected by the historical drone at the aforementioned location, and the movement path is the historical obstacle avoidance path made by the historical drone for the historical edge contour.

[0062] In summary, the technical solution provided in this application matches the edge contour detected by the UAV with the historical edge contour detected by a previous UAV at the same location. The historical obstacle avoidance path of the previous UAV with the highest contour similarity at that location for the reference obstacle is determined as the obstacle avoidance path of the UAV for the stationary obstacle. Then, the UAV is controlled to avoid the stationary obstacle based on this obstacle avoidance path. The obstacle avoidance path is determined based on previously recorded data, eliminating the need for real-time planning of obstacle avoidance paths for stationary obstacles. The path planning pressure during the UAV's movement is low, and the obstacle avoidance path is the successful obstacle avoidance path of the previous UAV. While reducing the amount of computation required for path planning, it can effectively ensure that the current obstacle avoidance operation is as successful as the previous obstacle avoidance operation, thereby improving the obstacle avoidance success rate and thus improving the movement efficiency of the UAV. Moreover, in the real-time obstacle avoidance process of the UAV, the previously saved historical data is given value, improving the utilization rate of historical data.

[0063] Furthermore, in the absence of historical drones with contour similarity greater than the similarity threshold, the nearest edge point is determined as the target point by measuring the distance between each edge point in the edge contour and the drone. This allows the drone to be controlled to avoid stationary obstacles by bypassing the target point. In other words, when obstacle avoidance control of the drone cannot be based on historical data, the drone is controlled to avoid stationary obstacles based on the real-time acquired edge contour, achieving automatic obstacle avoidance and improving the success rate of obstacle avoidance, which is beneficial to ensuring the smooth movement of the drone. Moreover, based on the stationary nature of the obstacle, obstacle avoidance is performed directly based on the edge contour, without the need to start other equipment or build a map model, resulting in low computational load.

[0064] Furthermore, after a drone successfully avoids a stationary obstacle, the correspondence between its movement path, position, and edge contour is recorded. This successful obstacle avoidance is recorded as historical data. When other drones encounter the same stationary obstacle in the future, they can directly avoid the obstacle based on this recorded historical data, without having to repeatedly plan paths for the same stationary obstacle. This improves the obstacle avoidance success rate and reduces the computational load required for path planning at the multi-drone level. Moreover, even if one or more new stationary obstacles are generated due to environmental changes, there is no need to update the 3D map through rescanning or other operations. The newly generated stationary obstacle is used as the basic unit, and the corresponding obstacle avoidance path is recorded when the drone passes through the new stationary obstacle. There is no need to scan the map in advance to build a 3D map, nor is there a need to detect environmental changes in real time to update the 3D map, reducing the computational load. It can also successfully avoid obstacles when the environment changes, thus improving the obstacle avoidance success rate.

[0065] Please refer to Figure 3 This illustrates a flowchart of a drone motion control method provided in another embodiment of this application. The method is applied to... Figure 1 The computer device 20 in the flight control system shown. The method may include the following steps (301-307): Step 301: Call the lidar to scan along the direction of the UAV's movement to obtain the first radar data.

[0066] Step 302: If it is determined that there is an obstacle in the direction of movement based on the first radar data, determine the type of obstacle based on the first radar data.

[0067] Step 303: If the obstacle is determined to be a stationary obstacle, the edge contour of the stationary obstacle is determined based on the first radar data.

[0068] Step 304: Control the drone to avoid stationary obstacles based on edge contours.

[0069] Steps 301-304 above and Figure 2 Steps 201-204 in the embodiment are similar; see details below. Figure 2 Examples are not detailed here.

[0070] Step 305: If the obstacle is determined to be a moving obstacle, the image acquisition device and the lidar are invoked to acquire at least two sets of dynamic data.

[0071] In this embodiment, the drone is equipped with an image acquisition device, such as a binocular camera, that has depth information acquisition capabilities. When the obstacle is determined to be a moving obstacle, the image acquisition device and a lidar are invoked to acquire at least two sets of dynamic data. Each set of dynamic data includes second radar data with the same timestamp, RGB (Red-Green-Blue) images, and depth information; different sets of dynamic data correspond to different timestamps.

[0072] Optionally, the second radar data may be the same as or different from the first radar data described above, and this application embodiment does not limit this.

[0073] In one possible implementation, the second radar data is the same as the first radar data. Optionally, to improve the real-time performance of obstacle avoidance for the UAV, the image acquisition device and the aforementioned lidar are activated simultaneously during the UAV's movement, and the image acquisition frequency and the laser emission frequency are kept consistent. After determining that the obstacle is a moving obstacle, the computer device acquires RGB images and depth information based on the timestamps corresponding to the data sets contained in the first radar data, thereby obtaining at least two sets of dynamic data.

[0074] In another possible implementation, the second radar data differs from the first radar data. Optionally, to conserve the drone's energy, the lidar is activated only during drone movement to acquire the first radar data and confirm the presence of an obstacle before the image acquisition device is activated. Subsequently, the image acquisition frequency and the laser emission frequency are kept consistent to acquire second radar data, RGB images, and depth information with the same timestamp, resulting in at least two sets of dynamic data.

[0075] In other possible implementations, the second radar data is partially identical to the first radar data. For example, while the UAV is moving, the image acquisition device and the aforementioned lidar are activated simultaneously, and the image acquisition frequency and the laser emission frequency are kept consistent. After determining that the obstacle is a moving obstacle, in order to further ensure the success rate of obstacle avoidance, the image acquisition device and lidar are used to continue data acquisition, obtaining at least two sets of dynamic data.

[0076] Step 306: Determine the speed and direction of motion of the moving obstacle based on at least two sets of dynamic data.

[0077] In this embodiment of the application, after acquiring the above-mentioned at least two sets of dynamic data, the computer device determines the speed and direction of movement of the moving obstacle based on the at least two sets of dynamic data.

[0078] Optionally, the computer device constructs at least two three-dimensional models of the moving obstacle based on at least two sets of dynamic data. Each three-dimensional model corresponds to one set of dynamic data. For example, after the UAV is equipped with the aforementioned lidar and image acquisition device, the lidar and image acquisition device are calibrated to obtain corresponding calibration parameters. Then, based on at least two sets of dynamic data and the calibration parameters, at least two three-dimensional models of the moving obstacle are constructed.

[0079] Optionally, after constructing the 3D model, the computer device acquires the posture change features and distance change features of the moving obstacle based on the at least two 3D models, and then predicts the movement trajectory of the moving obstacle based on the posture change features and the distance change features to obtain the movement speed and direction. For example, the computer device takes the posture change features, distance change features, and real-time environmental information as input, and obtains the movement speed and direction based on a pre-trained deep learning model. The environmental information includes, but is not limited to, at least one of the following: temperature, wind direction, humidity, weather, etc., which are not limited in this embodiment.

[0080] Step 307: Based on the speed and direction of movement, control the drone to avoid moving obstacles.

[0081] In this embodiment of the application, after obtaining the above-mentioned speed and direction of movement, the computer device controls the drone to avoid moving obstacles based on the speed and direction of movement.

[0082] Optionally, when the direction of movement is the same as the direction of motion, the computer device controls the drone's movement speed to be less than its speed of motion, until the aforementioned lidar can no longer detect the moving obstacle. Taking a two-dimensional top-down view as an example, such as... Figure 4 As shown, the direction of movement 42 of the moving obstacle 41 is the same as the direction of movement 44 of the drone 43. The computer device controls the movement speed of the drone 43 to be less than the movement speed of the moving obstacle 41, so that the drone 43 and the moving obstacle 41 will not collide.

[0083] Optionally, when the direction of motion is opposite to the direction of movement, the computer device determines the trajectory of the moving obstacle based on the direction of motion; and controls the drone to move outside the trajectory. Taking a two-dimensional top-down view as an example, such as... Figure 5As shown, the movement direction 51 of the moving obstacle 41 is opposite to the movement direction 52 of the drone 43. The computer device determines the movement trajectory 53 (white rectangular part) of the moving obstacle 41 based on the movement direction 51 of the moving obstacle 41, and further controls the drone 41 to move outside the movement trajectory 53.

[0084] Optionally, when the extensions of the motion direction and the movement direction intersect, the computer device determines the opposite direction of the motion direction as the new movement direction of the drone; and, based on the distance between the moving obstacle and the drone, determines the new movement speed of the drone; and, controls the movement of the drone based on the new movement direction and the new movement speed. Taking a two-dimensional top-down view as an example, such as... Figure 6 As shown, the movement direction 61 of the moving obstacle 41 intersects the extended movement direction 62 of the drone 43 at point 63. The computer device determines the new movement direction 65 of the drone based on the opposite direction 64 of the movement direction 61, and determines the new movement speed of the drone based on the distance between the moving obstacle 41 and the drone 43. Then, the computer controls the movement of the drone 43 based on the new movement direction 65 and the new movement speed. For example, to minimize the risk of collision between the moving obstacle and the drone, the distance between the moving obstacle and the drone is positively correlated with the new movement speed; that is, the greater the distance between the moving obstacle and the drone, the greater the new movement speed, and vice versa.

[0085] One point that needs to be clarified is that the above Figures 4-6 These are all illustrative examples of how drones avoid moving obstacles in a two-dimensional plane. In practical applications, if the drone and the moving obstacle do not collide in a two-dimensional plane, they will also not collide when they are at the same height. Therefore, even if the influence of height changes is added in a three-dimensional environment, the drone will still not collide with the moving obstacle if controlled based on the above description.

[0086] In summary, the technical solution provided in this application acquires RGB images and depth information through an image acquisition device, combines this with second radar data acquired by a lidar, determines the speed and direction of movement of moving obstacles, and then controls the drone to successfully avoid moving obstacles based on these speeds and directions, thereby improving the obstacle avoidance success rate and the overall smooth completion of the drone's movement. Moreover, by classifying obstacles into stationary and moving obstacles, a lower-energy-consumption method is used for stationary obstacles, relying solely on lidar for obstacle avoidance without calling the image acquisition device, while for moving obstacles, the image acquisition device is called to ensure the obstacle avoidance success rate. This makes the drone's energy allocation more reasonable and accurate, thereby improving the drone's overall endurance.

[0087] In addition, based on the different relationships between the direction of motion and the direction of movement, different obstacle avoidance schemes are assigned to the drone. This not only enriches the obstacle avoidance schemes, but also enables the drone to successfully avoid obstacles moving in different directions, thus improving the obstacle avoidance success rate.

[0088] Please refer to Figure 7 This illustrates a flowchart of a drone motion control method provided in another embodiment of this application. The method is applied to... Figure 1 The computer device 20 in the flight control system shown. The method may include the following steps (701-705): Step 701: Call the lidar to scan along the direction of the UAV's movement to obtain the first radar data.

[0089] Step 702: If it is determined that there is an obstacle in the direction of movement based on the first radar data, determine the type of obstacle based on the first radar data.

[0090] Step 703: If the obstacle is determined to be a stationary obstacle, the edge contour of the stationary obstacle is determined based on the first radar data.

[0091] Step 704: Control the drone to avoid stationary obstacles based on edge contours.

[0092] Steps 701-704 above and Figure 2 Steps 201-204 in the embodiment are similar; see details below. Figure 2 Examples are not detailed here.

[0093] Step 705: If a landing marker is detected by the image acquisition device, control the drone to land in the pre-set landing area.

[0094] In this embodiment of the application, in order to facilitate landing, the drone is equipped with an image acquisition device for detecting ground information. When the computer device detects a landing marker through the image acquisition device, it controls the drone to land in a pre-set landing area.

[0095] Optionally, after obtaining the landing identifier, the computer device determines the corresponding landing area based on the landing identifier, and allocates a corresponding landing point to the drone based on the vacant area in the landing area, thereby controlling the drone to land at the landing point.

[0096] For example, the landing identifier can be a QR code, a numerical mark, or the word "landing," etc., and this application embodiment does not limit this.

[0097] In summary, the technical solution provided in this application embodiment captures landing markers through an image acquisition device, thereby controlling the drone to land in a pre-set landing area. This helps ensure the successful landing of the drone and reduces energy consumption caused by the drone's inability to land.

[0098] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0099] Please refer to Figure 8 This diagram illustrates a block diagram of a mobile control device for a drone according to an embodiment of this application. The drone is equipped with a lidar system, and the device has the function of implementing the aforementioned drone mobile control method. This function can be implemented in hardware or by hardware executing corresponding software. The device can be the aforementioned computer equipment or can be installed within a computer equipment. The device may include: a radar scanning module 810, a type determination module 820, a contour detection module 830, and a stationary avoidance module 840.

[0100] The radar scanning module 810 is used to call the lidar to scan along the movement direction of the UAV and obtain radar data.

[0101] The type determination module 820 is used to determine the type of the obstacle based on the radar data when it is determined that there is an obstacle in the direction of movement based on the radar data.

[0102] The contour detection module 830 is used to determine the edge contour of the stationary obstacle based on the radar data when the obstacle is determined to be a stationary obstacle.

[0103] A stationary avoidance module 840 is used to control the UAV to avoid stationary obstacles based on the edge contour.

[0104] In an exemplary embodiment, such as Figure 9 As shown, the stationary avoidance module 840 includes: a history acquisition unit 841, a contour matching unit 842, a reference determination unit 843, a path determination unit 844, and a movement control unit 845.

[0105] The history acquisition unit 841 is used to acquire at least one historical edge contour detected by a historical drone at the location of the drone.

[0106] The contour matching unit 842 is used to match the edge contour with the historical edge contour to obtain the contour similarity corresponding to each of the historical edge contours.

[0107] The reference determination unit 843 is used to determine the historical drone with the largest contour similarity as the reference target among at least one historical drone with a contour similarity greater than a similarity threshold.

[0108] The path determination unit 844 is used to determine the historical obstacle avoidance path of the reference target at the location against the reference obstacle as the obstacle avoidance path of the UAV against the stationary obstacle.

[0109] The mobile control unit 845 is used to control the drone to avoid the stationary obstacle based on the obstacle avoidance path.

[0110] In an exemplary embodiment, such as Figure 9 As shown, the stationary avoidance module 840 further includes a distance acquisition unit 846 and a target determination unit 847.

[0111] The distance acquisition unit 846 is used to acquire the distance between each edge point in the edge contour and the drone when there is no historical drone with a contour similarity greater than the similarity threshold.

[0112] The target determination unit 847 is used to determine the nearest edge point as the target point.

[0113] The mobile control unit 845 is also used to control the drone to bypass the target point and avoid the stationary obstacle.

[0114] In an exemplary embodiment, such as Figure 9 As shown, the stationary avoidance module 840 also includes a data recording unit 848.

[0115] The data recording unit 848 is used to: acquire the movement path of the UAV; and record the correspondence between the movement path, the position, and the edge contour.

[0116] In an exemplary embodiment, the drone is equipped with an image acquisition device having depth information acquisition capabilities; such as Figure 9 As shown, the device also includes: a data acquisition module 850, a direction determination module 860, and a motion avoidance module 870.

[0117] The data acquisition module 850 is used to, when the obstacle is determined to be a moving obstacle, invoke the image acquisition device and the lidar to acquire at least two sets of dynamic data; wherein, one set of dynamic data includes second radar data with the same timestamp, RGB image and depth information, and different dynamic data correspond to different timestamps.

[0118] The direction determination module 860 is used to determine the speed and direction of movement of the moving obstacle based on the at least two sets of dynamic data.

[0119] The motion avoidance module 870 is used to control the drone to avoid the moving obstacle based on the motion speed and the motion direction.

[0120] In an exemplary embodiment, the motion avoidance module 870 is used for: When the direction of motion is the same as the direction of movement, the movement speed of the drone is controlled to be less than the speed of motion; When the direction of motion is opposite to the direction of movement, the trajectory of the moving obstacle is determined based on the direction of motion; and the drone is controlled to move outside the trajectory. When the direction of motion intersects the extended direction of movement, the opposite direction of the direction of motion is determined as the new direction of movement for the drone; and, based on the distance between the moving obstacle and the drone, a new speed of movement for the drone is determined; and, based on the new direction of movement and the new speed of movement, the drone is controlled to move.

[0121] In an exemplary embodiment, such as Figure 9 As shown, the drone is equipped with an image acquisition device for detecting ground information, and the device also includes a landing control module 880.

[0122] The landing control module 880 is used to control the drone to land in a pre-set landing area when the image acquisition device detects a landing marker.

[0123] In summary, the technical solution provided in this application uses LiDAR to scan the movement direction of the UAV and distinguish the type of obstacle by whether it is stationary. When it is determined that the detected obstacle is a stationary obstacle, the edge contour of the stationary obstacle is determined by the first radar data based on the characteristic that the obstacle is stationary. Then, the UAV is controlled to avoid the stationary obstacle based on the edge contour. There is no need to build a real-time map for obstacle avoidance, which can effectively reduce the amount of computation required for obstacle avoidance control of the UAV, thereby improving the movement efficiency of the UAV from the side.

[0124] In an exemplary embodiment, a computer device is also provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described mobile control method for the unmanned aerial vehicle.

[0125] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the above-described mobile control method for the unmanned aerial vehicle.

[0126] In an exemplary embodiment, a computer program product is also provided, which, when run, causes a computer device to execute the aforementioned drone motion control method.

[0127] The above embodiments are merely illustrative of the technical concept and features of the present invention, intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and should not be construed as limiting the scope of protection of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the present invention is defined by the appended claims rather than the foregoing description, and thus all changes falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention.

Claims

1. A method for controlling the movement of an unmanned aerial vehicle (UAV), characterized in that, The drone is equipped with a lidar, and the method includes: The lidar is invoked to scan along the movement direction of the UAV to obtain first radar data; If an obstacle is determined to exist in the direction of movement based on the first radar data, the type of the obstacle is determined based on the first radar data; If the obstacle is determined to be a stationary obstacle, the edge contour of the stationary obstacle is determined based on the first radar data; Controlling the drone to avoid the stationary obstacle based on the edge contour includes: Based on the location of the drone, obtain at least one historical edge profile detected by a historical drone at that location; Place the edge contour and the historical edge contour in the same three-dimensional coordinate system, and match the historical edge contour. The matching only includes translation and rotation operations to make the historical edge contour coincide with the edge contour. The matching does not include scaling operations. The contour similarity of each historical edge contour is determined based on the degree of overlap between the historical edge contours and the edge contours. In at least one historical UAV with a contour similarity greater than a similarity threshold, the historical UAV with the largest contour similarity is determined as a reference target, and the obstacle corresponding to the historical edge contour with the largest contour similarity is determined as a reference obstacle. The historical obstacle avoidance path of the reference target at the location for the reference obstacle is directly determined as the obstacle avoidance path of the UAV for the stationary obstacle. The drone is controlled to avoid stationary obstacles based on the obstacle avoidance path.

2. The method according to claim 1, characterized in that, After determining the contour similarity corresponding to each of the historical edge contours, the method further includes: In the absence of any historical drones with a contour similarity greater than the similarity threshold, the distance between each edge point in the edge contour and the drone is obtained; The nearest edge point is identified as the target point; Control the drone to bypass the target point and avoid the stationary obstacle.

3. The method according to claim 2, characterized in that, After controlling the drone to bypass the target point and avoid the stationary obstacle, the method further includes: Obtain the movement path of the drone; Record the correspondence between the movement path, the position, and the edge contour.

4. The method according to claim 1, characterized in that, The drone is equipped with an image acquisition device with depth information acquisition function; After determining the type of obstacle based on the first radar data when it is determined that an obstacle exists in the direction of movement based on the first radar data, the method further includes: If the obstacle is determined to be a moving obstacle, the image acquisition device and the lidar are invoked to acquire at least two sets of dynamic data; wherein, one set of dynamic data includes second radar data with the same timestamp, RGB image and depth information, and different dynamic data correspond to different timestamps; The speed and direction of motion of the moving obstacle are determined based on the at least two sets of dynamic data. Based on the speed and direction of movement, the drone is controlled to avoid the moving obstacle.

5. The method according to claim 4, characterized in that, The method of controlling the drone to avoid the moving obstacle based on the movement speed and the movement direction includes: When the direction of motion is the same as the direction of movement, the movement speed of the drone is controlled to be less than the speed of motion; When the direction of motion is opposite to the direction of movement, the trajectory of the moving obstacle is determined based on the direction of motion; and the drone is controlled to move outside the trajectory. When the direction of motion intersects the extended direction of movement, the opposite direction of the direction of motion is determined as the new direction of movement for the drone; and, based on the distance between the moving obstacle and the drone, a new speed of movement for the drone is determined; and, based on the new direction of movement and the new speed of movement, the drone is controlled to move.

6. The method according to any one of claims 1 to 5, characterized in that, The drone is equipped with an image acquisition device for detecting ground information, and the method further includes: If a landing marker is detected by the image acquisition device, the drone is controlled to land in a pre-set landing area.

7. A mobile control device for an unmanned aerial vehicle (UAV), characterized in that, The drone is equipped with a lidar system, and the drone's motion control device is used to implement the method as described in any one of claims 1 to 6, the device comprising: The radar scanning module is used to call the lidar to scan along the movement direction of the UAV and obtain radar data; A type determination module is used to determine the type of obstacle based on the radar data when it is determined that there is an obstacle in the direction of movement based on the radar data. A contour detection module is used to determine the edge contour of a stationary obstacle based on radar data when the obstacle is determined to be a stationary obstacle. A stationary avoidance module is used to control the UAV to avoid stationary obstacles based on the edge contour.

8. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the method as claimed in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images