Obstacle avoidance method for unmanned aerial vehicle, storage medium, task machine and unmanned aerial vehicle system

Abstract

The application discloses an obstacle avoidance method of a UAV, a storage medium, a task machine and a UAV system, and relates to the technical field of UAVs. The method comprises the following steps: acquiring a front view image and flight attitude data of a UAV in real time, and performing target detection on each front view image by using a pre-trained target recognition model to obtain a target detection result; in the case that the target detection result is that an obstacle exists, performing tracking on the obstacle by using a preset target tracking algorithm to obtain a tracking result of the obstacle; judging whether the obstacle is a target obstacle based on the tracking result of the obstacle, the center point coordinates of the front view image and the area proportion of the obstacle in the front view image, and in the case that the obstacle is the target obstacle, determining an obstacle avoidance path based on the tracking result of the target obstacle and the flight attitude data, so that the UAV performs obstacle avoidance according to the obstacle avoidance path. Thus, the problems of poor real-time performance and inaccurate obstacle avoidance in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, and particularly relates to an obstacle avoidance method of an unmanned aerial vehicle, a storage medium, a task machine and an unmanned aerial vehicle system. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology, its application in the fields of logistics, inspection and agriculture is increasingly widespread. However, the autonomous obstacle avoidance capability of unmanned aerial vehicles in complex environments still faces many challenges. Existing obstacle avoidance technologies mainly rely on multi-sensor fusion (such as vision, laser radar and ultrasonic wave) and obstacle avoidance strategies based on rules or machine learning.

[0003] However, the current obstacle avoidance strategy based on machine learning has high false detection rate and missed detection rate, and the obstacle avoidance strategy is single, which leads to inaccurate and low-efficiency obstacle avoidance of unmanned aerial vehicles. SUMMARY

[0004] The main purpose of the present application is to provide an obstacle avoidance method of an unmanned aerial vehicle, a storage medium, a task machine and an unmanned aerial vehicle system, so as to solve the problems of poor real-time performance and inaccurate obstacle avoidance in the prior art.

[0005] To achieve the above purpose, the present application provides an obstacle avoidance method of an unmanned aerial vehicle, comprising:

[0006] real-time acquisition of front view images and flight attitude data of the unmanned aerial vehicle, and target detection on each front view image by using a pre-trained target recognition model to obtain a target detection result;

[0007] In the case that the target detection result is that there is an obstacle, a preset target tracking algorithm is used to track the obstacle to obtain a tracking result of the obstacle;

[0008] Based on the tracking result of the obstacle, the center point coordinates of the front view image and the area proportion of the obstacle in the front view image, it is judged whether the obstacle is a target obstacle, and in the case that the obstacle is the target obstacle, an obstacle avoidance path is determined based on the tracking result of the target obstacle and the flight attitude data, so that the unmanned aerial vehicle performs obstacle avoidance according to the obstacle avoidance path.

[0009] Optionally, the tracking result of the obstacle comprises coordinate data of the obstacle in each frame of the front view image; and the determining whether the obstacle is a target obstacle based on the tracking result of the obstacle, the center point coordinate of the front view image and the area proportion of the obstacle in the front view image comprises: determining whether the obstacle moves towards the center point of the front view image based on the coordinate data of the obstacle in each frame of the front view image and the center point coordinate of the front view image; and if it is determined that the obstacle moves towards the center point of the front view image or the area proportion of the obstacle in the front view image is greater than a preset proportion, the obstacle is regarded as the target obstacle.

[0010] Optionally, the determining whether the obstacle moves towards the center point of the front view image based on the coordinate data of the obstacle in each frame of the front view image and the center point coordinate of the front view image comprises: determining a plurality of displacement amounts of the obstacle based on the coordinate data of the obstacle in each frame of the front view image, the displacement amount being a displacement amount of the obstacle between a current frame of the front view image and a next frame of the front view image; determining a moving speed of the obstacle based on each displacement amount; and determining whether the obstacle moves towards the center point of the front view image by using a preset formula and based on the moving speed of the obstacle and the center point coordinate of the front view image.

[0011] Optionally, the preset formula is:

[0012]

[0013] wherein, θ is an included angle between the moving speed and a center point vector; is the moving speed of the obstacle; is the moving speed of the obstacle; is the center point vector; is the center point coordinate of the front view image; and a and c are velocities in x and y directions respectively.

[0014] Optionally, the moving speed of the obstacle is calculated by the following formula:

[0015]

[0016] wherein, θ is an included angle between the moving speed and a center point vector; is the moving speed of the obstacle; is a coordinate of the obstacle in a t frame of the front view image; is a coordinate of the obstacle in a t+1 frame of the front view image; and T is a preset total frame number of the front view image.

[0017] Optionally, the training step of the target recognition model comprises: obtaining a training data set comprising a plurality of sample images with labeled boxes and labeled information, and processing each of the sample images using Mosaic data enhancement and adaptive image scaling; obtaining target anchor boxes based on the labeled boxes in each of the sample images and using a preset adaptive anchor box algorithm; inputting each of the processed sample images into a pre-constructed target recognition model, the target recognition model obtaining a target recognition result based on the target anchor boxes, and calculating a loss value based on the target recognition result and the labeled information using a CIOU loss function, updating parameters of the target recognition model based on the loss value until the target recognition model converges, and obtaining the trained target recognition model.

[0018] Optionally, the obtaining of the target anchor boxes based on the labeled boxes in each of the sample images and using a preset adaptive anchor box algorithm comprises: in a case where a recall rate of initial anchor point boxes of the target recognition model is lower than a preset threshold, randomly initializing N anchor point boxes, wherein N is a positive integer; updating each of the anchor point boxes using a genetic algorithm, a clustering algorithm, each of the labeled boxes in each of the sample images, and each of the anchor point boxes until each of the anchor point boxes satisfies a preset condition, and then taking each of the anchor point boxes as the target anchor box, the preset condition being that the recall rate of each of the anchor point boxes is higher than the preset threshold.

[0019] In addition, to achieve the above-mentioned purposes, the present application also provides a task machine of a UAV, comprising: an acquisition module, configured to acquire real-time front-view images and flight attitude data of a UAV; a target recognition module, configured to perform target detection on each of the front-view images using a pre-trained target recognition model to obtain a target detection result; a target tracking module, configured to, in a case where the target detection result is that there is an obstacle, track the obstacle using a preset target tracking algorithm to obtain a tracking result of the obstacle; and an obstacle avoidance module, configured to determine whether the obstacle is a target obstacle based on the tracking result of the obstacle, a center point coordinate of the front-view image, and an area proportion of the obstacle in the front-view image, and in a case where the obstacle is the target obstacle, determine an obstacle avoidance path based on the tracking result of the target obstacle and the flight attitude data, so that the UAV performs obstacle avoidance according to the obstacle avoidance path.

[0020] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the main network coordination voltage regulation method according to any one of the above.

[0021] The application further provides an unmanned aerial vehicle system, comprising the task machine, the camera and the flight control chip as described above, the task machine, the camera and the flight control chip realize data interaction by using ros1; the camera is used for acquiring front view images of the unmanned aerial vehicle in real time and sending the front view images to the task machine; the flight control chip is used for acquiring flight attitude data of the unmanned aerial vehicle and sending the flight attitude data to the task machine; the task machine is used for receiving the front view images and the flight attitude data of the unmanned aerial vehicle in real time, performing target detection on each front view image by using a pre-trained target recognition model to obtain a target detection result, performing tracking on an obstacle in the case that the target detection result is that the obstacle exists by using a preset target tracking algorithm to obtain a tracking result of the obstacle, judging whether the obstacle is a target obstacle based on the tracking result of the obstacle, a center point coordinate of the front view image and an area proportion of the obstacle in the front view image, and determining an obstacle avoidance path based on the tracking result of the target obstacle and the flight attitude data in the case that the obstacle is the target obstacle, so that the unmanned aerial vehicle performs obstacle avoidance according to the obstacle avoidance path.

[0022] The obstacle avoidance method of the unmanned aerial vehicle of the application can effectively screen and identify the target obstacle which may cause obstacles to the unmanned aerial vehicle, reduce the missed detection rate and the false detection rate, and further improve the obstacle avoidance accuracy of the unmanned aerial vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 FIG. 1 is a schematic diagram of an unmanned aerial vehicle system according to an embodiment of the application;

[0024] Figure 2 FIG. 2 is a flowchart of an obstacle avoidance method according to an embodiment of the application;

[0025] Figure 3 FIG. 3 is a framework diagram of a target recognition model according to an embodiment of the application;

[0026] Figure 4 FIG. 4 is another flowchart of an obstacle avoidance method according to an embodiment of the application;

[0027] Figure 5 FIG. 5 is a structural schematic diagram of a task machine according to an embodiment of the application;

[0028] In the figure, 500, task machine; 510, acquisition module; 520, target identification module; 530, target tracking module; 540, obstacle avoidance module.

[0029] The implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0030] To make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the accompanying drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0031] For the convenience of understanding, the unmanned aerial vehicle system of the embodiments of the present application will be introduced in detail first. Figure 1 is a schematic diagram of the unmanned aerial vehicle system of the embodiments of the present application.

[0032] As Figure 1 indicated, the unmanned aerial vehicle system can include a task machine, a camera and a flight control chip. Among them, the task machine, the camera and the flight control chip realize data interaction by using ros1, the camera is used to acquire real-time front view images of the unmanned aerial vehicle and send the front view images to the task machine; the flight control chip is used to acquire flight attitude data of the unmanned aerial vehicle and send the flight attitude data to the task machine; the task machine is used to receive real-time front view images and flight attitude data of the unmanned aerial vehicle, and use a pre-trained target identification model to detect targets in each front view image to obtain a target detection result, in the case that the target detection result is that there is an obstacle, use a preset target tracking algorithm to track the obstacle to obtain a tracking result of the obstacle, based on the tracking result of the obstacle, the center point coordinates of the front view image and the area proportion of the obstacle in the front view image, judge whether the obstacle is a target obstacle, in the case that the obstacle is a target obstacle, determine an obstacle avoidance path based on the tracking result of the target obstacle and the flight attitude data, so that the unmanned aerial vehicle avoids obstacles according to the obstacle avoidance path.

[0033] In the embodiment, the unmanned aerial system framework can be built based on three core frameworks of ROS1 (Robot Operating System version 1), TensorRT and PX4 flight control framework. Among them, the task machine can include an acquisition module, a target recognition module, a target tracking module and an obstacle avoidance module, the task machine internally adopts the ROS1 node mode to realize communication, and then realizes the front view image acquisition, target recognition, target tracking and obstacle avoidance, and each module of the task machine adopts the ROS1 node architecture. It should be noted that ROS1 supports three types of communication modes of topics, services and actions.

[0034] Further, the deep learning part in the task machine is realized by the TensorRT framework. In actual application process, the pre-trained target recognition model can be converted into an Engine file that can be accelerated by the hardware of the task machine, the target recognition module obtains each front view image through the ROS1 node, and converts each front view image into a Tensor tensor recognizable by TensorRT, and inputs into the target recognition model to obtain the target recognition result. After obtaining the target recognition result, the target recognition module converts the target recognition result into a ROS1 topic for publishing, so as to be called and read by the nodes of each module.

[0035] The camera used in the embodiment can be a finished FPV monocular camera, which provides a corresponding interface to obtain real-time front view images. The PX4 flight control framework can be deployed on a flight control chip supporting PX4, and the flight control chip is deployed in the unmanned aerial vehicle.

[0036] In addition, the communication between the ROS1 node in the task machine and the PX4 flight control chip can be realized by using the mavros / mavlink protocol, the mavros converts the ROS1 topic into the mavlink protocol information, and then the mavlink protocol information is transmitted to the mavlink service in the flight control chip through physical connection, and the service is parsed. Similarly, the flight control chip can also transmit data to the mavros of the task machine through the mavlink protocol, and then convert it into the ros1 topic, which is called and read by the nodes of each module.

[0037] For example, the hardware environment of the task machine can be Nvidia Nano 0.5Tops, and the software environment can be ubuntu20.04. The software environment of the flight control chip can be an Arm system.

[0038] The unmanned aerial vehicle system of the embodiment adopts a modular design idea. The visual sensor module, the radar sensor module and the task machine are integrated through a support, installed at corresponding positions of the unmanned aerial vehicle, and physically connected to the flight control chip to achieve the effect of sensing and obstacle avoidance. It should be noted that the integrated hardware can be installed at the head position of the unmanned aerial vehicle, so as to facilitate the timely acquisition of front view images and improve the accuracy of obstacle avoidance.

[0039] With reference to the contents in the above unmanned aerial vehicle system embodiments, the obstacle avoidance method of the unmanned aerial vehicle of the embodiment of the application will be described in detail below.

[0040] Figure 2 is one of the flowcharts of the obstacle avoidance method of the embodiment of the application. The obstacle avoidance method can be executed by the task machine in the foregoing embodiments. As shown in Figure 2 , the obstacle avoidance method can include the following steps:

[0041] Step 210: Real-time acquisition of front view images and flight attitude data of the unmanned aerial vehicle, and target detection on each front view image by using a pre-trained target recognition model to obtain a target detection result.

[0042] First of all, it should be noted that the front view image of the unmanned aerial vehicle is image data acquired by the unmanned aerial vehicle along the motion axis direction (usually consistent with the head direction) during flight.

[0043] In the embodiment, the task machine can acquire the front view images from the FPV camera of the unmanned aerial vehicle system in real time. Specifically, the FPV camera can capture the front view images of the unmanned aerial vehicle in real time, and the task machine can obtain each front view image in real time through the interface of the FPV camera. After the task machine acquires each front view image, each front view image can be converted into an image topic that can be recognized by a ROS1 node and published to each module of the task machine. Considering the frame rate of the FPV camera itself, the frequency of publishing the image topic by the ROS1 node should be consistent with the frame rate of the FPV camera.

[0044] In addition, the task machine can acquire the flight attitude data of the unmanned aerial vehicle from the flight control chip. It should be noted that the flight attitude data can include GPS latitude and longitude information (longitude, latitude), barometric height (relative height from the ground), unmanned aerial vehicle attitude information (heading angle, pitch angle, roll angle), unmanned aerial vehicle motion information (speed, acceleration). The flight control chip of the foregoing embodiment can acquire the above-mentioned flight attitude data, and then utilize a ROS1 node in combination with a mavros protocol and a mavlink protocol to transmit the flight attitude data to the task machine. The flight attitude data can be used to analyze the flight state of the current unmanned aerial vehicle, and then facilitate the subsequent determination of an obstacle avoidance strategy.

[0045] After the task machine obtains the front view image, the task machine will pre-process the front view image. The pre-processing can include scaling processing, normalization, etc., so as to scale and process the front view image to be able to be input into the pre-trained target recognition model. The size of the processed front view image is 608*608. The processed front view image is input into the trained target recognition model. The target recognition model can identify the object category in the front view image and obtain the object position (i.e., the target recognition result).

[0046] In some embodiments, the target recognition model can adopt the existing YOLOv5 model. The training steps of the target recognition model can include: obtaining a training data set, the training data set including a plurality of sample images with labeled boxes and labeled information, and processing each sample image by using Mosaic data enhancement and adaptive image scaling; obtaining a target anchor box based on the labeled box in each sample image and by using a preset adaptive anchor box algorithm; inputting each processed sample image into a pre-constructed target recognition model, the target recognition model obtaining a target recognition result based on the target anchor box, and calculating a loss value by using a CIOU loss function based on the target recognition result and the labeled information, updating the parameters of the target recognition model based on the loss value, until the target recognition model converges, and obtaining the trained target recognition model.

[0047] It should be noted that the training data set of the present application can include a plurality of sample images. The sample images can be any images with objects, and the sample images have labeled information and corresponding labeled boxes for each object.

[0048] The training method of the target recognition model provided in the present embodiment is different from the conventional training method of the target recognition model. The present embodiment uses the Mosaic data enhancement method to process the training samples, and applies the adaptive anchor box calculation method and the adaptive image scaling method in the model training process, thereby improving the training speed of the model and the accuracy of the network.

[0049] Specifically, after obtaining the training data set, the Mosaic data enhancement method can be used to perform data enhancement on the training data set. The Mosaic data enhancement method for processing the training samples specifically includes: splicing four sample images in a random scaling, random cropping and random arrangement manner to obtain a new sample image. This enhancement method can combine several images into one image, which not only enriches the training data set, but also greatly improves the training speed of the network, and reduces the memory requirement of the model.

[0050] Further, the adaptive picture scaling method is an improvement of the original image scaling. In the original image scaling algorithm, the sample image is directly scaled. If the aspect ratio before and after scaling is inconsistent, the imaging result will be distorted. The adaptive picture scaling method adds black edges to the original image before scaling to ensure that the aspect ratio of the sample image before and after scaling is consistent, thereby ensuring that the imaging result is consistent.

[0051] In this embodiment, the Mosaic data enhancement method and the adaptive picture scaling method described above can be used to preprocess the obtained training sample set to obtain a new training sample set, and then the target recognition model is trained using the new training sample set.

[0052] During the training of the target recognition model, the target anchor frame can also be obtained based on the annotation frame in each sample image and by using a preset adaptive anchor frame algorithm, so that the target recognition model uses the adaptive target anchor frame for recognition and training.

[0053] It should be noted that the anchor frame in this embodiment is an anchor point frame. The anchor point frame refers to a group of predefined boundary frames, which are generally consistent with the width and height of the target object in each sample image in terms of size and aspect ratio. These frames have different aspect ratios and scales to cover the width and height of the target object in the training data set as much as possible. The adaptive anchor frame is calculated adaptively according to different input training data sets during the training process to obtain the most suitable anchor point frame result of the training data set.

[0054] In some embodiments, obtaining the target anchor frame based on the annotation frame in each sample image and by using a preset adaptive anchor frame algorithm includes: in a case where the recall rate of the initial anchor point frame of the target recognition model is lower than a preset threshold, randomly initializing N anchor point frames, where N is a positive integer; updating each anchor point frame by using a genetic algorithm, a clustering algorithm, each annotation frame in each sample image, and each anchor point frame until each anchor point frame satisfies a preset condition, and then taking each anchor point frame as the target anchor frame, and the preset condition is that the recall rate of each anchor point frame is higher than the preset threshold.

[0055] Specifically, first, the target recognition model calculates the best recall rate of the initial anchor point frame. When the best recall rate is lower than the preset threshold, the anchor point needs to be recalculated until the best recall rate is higher than the preset threshold to obtain a new anchor point frame (i.e., the target anchor frame). In this embodiment, the preset threshold can be set by the staff according to actual needs. For example, the preset threshold can be set to 0.98. The best recall rate can be calculated by the following formula:

[0056]

[0057]

[0058] where w and h are the width and height of the bounding box in the training dataset, respectively; and are the width and height of the current anchor box, respectively; is the number of bounding boxes in the training dataset; is an indicator function; is a threshold value (default is 4.0); bpr is the best recall rate; r is an intermediate variable.

[0059] If the best recall rate (i.e., recall rate) of the initial anchor box calculated by the target recognition model is lower than the preset threshold value, the target recognition model recalculates the anchor point.

[0060] The YOLOv5 model used in this embodiment uses a K-means clustering algorithm to calculate the anchor point. Specifically, the target recognition model randomly initializes N anchor points, and for each bounding box in the training dataset, calculates the distance between it and each anchor point, thereby clustering the bounding boxes, and updating the position of the anchor points according to the clustering result to obtain new anchor boxes. The mathematical formula is as follows:

[0061]

[0062]

[0063] where, is the distance between the bounding box i and the anchor point j; is the number of bounding boxes in the cluster j; is the set of bounding boxes in the cluster j; is the new anchor box; and represent the width and height of the bounding box i, and represent the width and height of the new anchor box .

[0064] In this embodiment, in order to further optimize the anchor point, the YOLOv5 model can also use a genetic algorithm to optimize the anchor point. The specific steps are as follows: use a fitness function to evaluate the performance of each randomly generated anchor box, and select and mutate the anchor point according to the fitness. Combining the above formula for the best recall rate and the K-means clustering calculation formula, the fitness function can be derived as follows:

[0065]

[0066] where, is the fitness of the anchor point k; N is the number of bounding boxes; is an indicator function; is a threshold value (default is 4.0).

[0067] Therefore, the genetic algorithm and clustering algorithm are repeatedly executed until the new anchor box obtained meets the requirement that the optimal recall rate is higher than the preset threshold, and the target anchor box can be obtained.

[0068] In this embodiment, the baseline network of the target recognition model (i.e., the YOLOv5 model) is typically a network of a high-performance classifier, which is used to extract some general feature representations. The YOLOv5 model uses not only the CSPDarknet53 structure but also the Focus structure as the baseline network.

[0069] The CSPDarknet53 architecture primarily consists of convolutional layers (Conv), batch normalization (BN) layers, LeakReLu activation layers, cross-stage partial connections (CSP), and spatial pyramid pooling (SPPF). These components are then processed using... Figure 3 The structures within the model are connected to form its main framework.

[0070] The mathematical expression for convolution is as follows:

[0071]

[0072] Where I is the input feature map with dimensions H*W*C, and K is the convolution kernel with dimensions H*W*C*K. This is the output feature map, with dimensions H*W*K. 'c' represents the data for each layer along dimension C.

[0073] The mathematical expression for the BN layer is as follows:

[0074]

[0075] In the formula, It is the first of the input feature maps One sample; It is the batch average; the calculation formula is: ; It is the batch variance, calculated using the following formula: ; and These are learnable scaling and offset parameters; It is a very small constant used to ensure numerical stability.

[0076] Furthermore, the formula for the LeakReLu activation layer is: .

[0077] The input feature map is split into two parts: and ,in After convolution operations and residual connections Only convolution operation is needed, and the formula is as follows:

[0078]

[0079] The spatial pyramid pooling (SPPF) is an input feature map F that undergoes different scale pooling operations and is spliced, and its mathematical expression is as follows:

[0080]

[0081] In the formula, represents taking the maximum value in the range of size k to fill the entire range; is a different pooling kernel size, represents that the feature map and are connected together.

[0082] The Focus structure is a network structure based on slicing operation, which can effectively convert high-resolution input feature maps into low-resolution feature maps with increased channel numbers by downsampling the input image through slicing operation. In mathematical language, for an input tensor, the Focus structure slices it into small blocks and splices along the channel dimension to obtain an output tensor. Its mathematical expression can be:

[0083]

[0084]

[0085] In the formula, and are the row and column indices of the input tensor; and are the row and column indices of the small slice block, ranging from 0 to ; is the channel index of the input tensor; is the channel index of the output tensor, ranging from 0 to . This reorganization allows the Focus structure to rearrange the spatial information of the input image into the channel, so that it can process multi-scale information in the subsequent convolution layer and reduce the amount of calculation and parameters.

[0086] Further, the Neck network of the target recognition model (i.e., the YOLOv5 model) is usually located in the middle position between the benchmark network and the head network, which can further improve the diversity and robustness of the features. Based on a large number of convolution operations, the ability of network feature fusion is enhanced.

[0087] The head network is used to complete the output of the target recognition result. The number of branches of the output end is different for different detection algorithms, and usually includes a classification branch and a regression branch. Among them, CIOU as a new loss function is applied to the model to improve the accuracy of the model. The formula is as follows:

[0088]

[0089]

[0090] In the formula, Distance_C represents the distance of the diagonal line between the prediction result and the true result; Distance_2 represents the distance between the center points of the prediction result and the true result; and represent the width and height of the true result; and represent the width and height of the prediction result; IOU represents the value obtained by dividing the intersection part of the prediction result and the true result by the union of the prediction result and the true result.

[0091] Therefore, the target recognition model trained in the above manner is used to recognize the obstacles in the front view image, the robustness of the target recognition model is improved, and the accuracy of the target recognition result is improved.

[0092] Step 220: In the case that the target detection result is that there is an obstacle, a preset target tracking algorithm is used to track the obstacle to obtain a tracking result of the obstacle.

[0093] It should be noted that since the obstacles appearing in the front view image are not necessarily obstacles that need to be avoided, the obstacles need to be tracked to further determine whether the obstacles are obstacles that need to be avoided.

[0094] In the embodiment, after the task machine obtains the target recognition result, the target recognition result can be used to track the obstacle to obtain the motion trajectory of the obstacle in the front and rear frames of the front view image, so as to facilitate subsequent estimation of the relative motion between the unmanned aerial vehicle and the obstacle, and further determination of whether the unmanned aerial vehicle has a collision risk.

[0095] The preset target tracking algorithm of the embodiment of the application can be a bytetrack algorithm. The bytetrack algorithm is a tracking method based on the tracking-by-detection paradigm, which is based on the detection results of the front and rear frames of the front view image to construct the association relationship between the front and rear frames of the obstacle, and is used to generate a target ID. Most multi-target tracking methods obtain the target ID by detecting the frame whose association score is higher than a threshold. For targets with low detection scores, such as occluded targets, they are simply discarded, which brings non-negligible problems, including a large number of missed detections and fragmented trajectories.

[0096] To solve this problem, the present embodiment also adopts BYTE association algorithm, which tracks by associating each detection box instead of only high-score detection boxes. For low-score detection boxes, their similarity to trajectories is utilized to recover real obstacles and filter out background detections.

[0097] Specifically, first, detection boxes and corresponding detection scores are obtained by a detector, and the detection boxes are classified, if the score is higher than T_high, the detection box is classified into high-confidence group, if the score is lower than T_high and higher than T_low, the detection box is classified into low-confidence group. Here the detector is the target recognition model in the foregoing embodiment, and the detection box score can be directly provided by the target recognition model.

[0098] Further, the matching process uses the similarity between the detection box and the Kalman filter estimation result, here IoU or Re-ID feature distance can be used as the similarity measure. Then based on the similarity, the Hungarian algorithm is used for matching, and those high-confidence detection boxes that are not matched to the trajectory and trajectories that are not matched to the detection box are retained.

[0099] Wherein, the feature distance is calculated as follows:

[0100]

[0101] In the formula, is the feature distance; is the predicted box of the trajectory; is the detection box; is the intersection over union of the two boxes.

[0102] The trajectory prediction process based on Kalman filter is as follows, first assume that the state vector of the trajectory is The prediction step is state prediction and covariance prediction, wherein the state prediction is:

[0103]

[0104] In the formula, is the state transition matrix, is the predicted state at time k, is the estimated state at time k-1.

[0105] The covariance prediction is:

[0106]

[0107] In the formula, is the prediction error covariance; is the process noise covariance. The mathematical calculation that is carried out later is to use the Hungarian algorithm to perform linear assignment and find the optimal match, which is mathematically expressed as:

[0108]

[0109] The matching is to match the newly obtained detection frame with the detection frame obtained through Kalman prediction. The matched data can be used as an observation value and input to the update part of Kalman, including state update and covariance update, to achieve the effect of correcting the prediction model. The mathematical expression of the state update is as follows:

[0110]

[0111] wherein, is the detection frame observation value, is the observation matrix, is the Kalman gain. The covariance update is: wherein, I is the unit matrix.

[0112] Further, associate those trajectories left after the first association and low confidence detection frames. Then keep those trajectories that are still not matched to the bounding box after the second matching, and delete those low confidence bounding boxes that do not find the corresponding trajectory after the second matching, because these bounding boxes are identified as background that does not contain any object.

[0113] Finally, save those high confidence bounding boxes that are not matched to the corresponding trajectory as a newly appearing trajectory. Initialize the detection frames that are not matched in both matches into new trajectories.

[0114] The BYTE data association method is different from other methods that only associate high confidence detection frames. It almost retains all detection frames and performs identity matching, and divides them into two groups of high confidence and low confidence. First, associate the high confidence detection frames into the trajectory, and then associate the low confidence detection frames with the unmatched tracking objects to retain the low confidence detection frames and filter the background. Thus, the byte track target tracking algorithm can be used to more accurately obtain the motion trajectory of each obstacle.

[0115] Step 230: Based on the tracking result of the obstacle, the center point coordinates of the front view image, and the area ratio of the obstacle in the front view image, it is judged whether the obstacle is a target obstacle, and in the case that the obstacle is a target obstacle, the obstacle avoidance path is determined based on the tracking result of the target obstacle and the flight attitude data, so that the unmanned aerial vehicle performs obstacle avoidance according to the obstacle avoidance path.

[0116] It should be noted that the center point coordinates of the front view image and the area ratio of the obstacle in the front view image can be obtained by the task machine itself.

[0117] The embodiment provides two methods for determining whether the obstacle is a target obstacle, one of which is directly using the tracking result (i.e., the motion trajectory) of the obstacle to determine whether the obstacle is a target obstacle, and the other of which is combining the area ratio of the obstacle in the front view image to determine whether the obstacle is a target obstacle. The two determination methods can be used simultaneously, so as to further improve the detection accuracy of the target obstacle.

[0118] The target obstacle determination method and the obstacle avoidance strategy of the embodiment are described below in detail.

[0119] In some embodiments, the tracking result of the obstacle includes coordinate data of the obstacle in each frame of the front view image and a motion trajectory of the obstacle. Figure 4 FIG. 2 is a flowchart of the obstacle avoidance method of the embodiment. Figure 4 As shown in FIG. 2, step 230 determines whether the obstacle is a target obstacle based on the tracking result of the obstacle, the center point coordinate of the front view image, and the area ratio of the obstacle in the front view image, which can include the following steps:

[0120] Step 410: determines whether the obstacle moves toward the center point of the front view image based on the coordinate data of the obstacle in each frame of the front view image and the center point coordinate of the front view image.

[0121] Step 420: if it is determined that the obstacle moves toward the center point of the front view image or the area ratio of the obstacle in the front view image is greater than a preset ratio, the obstacle is regarded as a target obstacle.

[0122] It should be noted that the coordinate data of the obstacle in each frame of the front view image can be coordinate data of a center point of the obstacle.

[0123] It can be understood that, in the flight process, the relative motion between the obstacle and the aircraft needs to be considered. Considering that the FPV camera is installed in the head portion of the unmanned aerial vehicle and provides the front view field of the unmanned aerial vehicle, if the obstacle can collide with the unmanned aerial vehicle, the motion trend of the obstacle in the image must approach the center of the image, and the size of the obstacle in the image must also meet certain requirements. In addition, considering the case that some obstacles suddenly appear from the outside and the tracking is lost, some independent targets in the image that meet certain area ratios and have no motion trajectory should also be determined as obstacles that need to be avoided.

[0124] Based on this, the embodiments of the present application determine whether the obstacle moves towards the center point of the front view image through the relationship between the center point of the obstacle and the center point of the front view image. If yes, it indicates that the obstacle is gradually approaching the UAV and has a high probability of collision with the UAV. If not, it indicates that the obstacle is not gradually approaching the UAV and may not collide with the UAV.

[0125] Meanwhile, the area ratio of the obstacle in the front view image can also be used for judgment. Specifically, if the area ratio of the obstacle is greater than a preset ratio, it indicates that the obstacle is close to the UAV, and thus has a high probability of collision with the UAV. If the area ratio of the obstacle is less than or equal to the preset ratio, it indicates that the obstacle is far from the UAV, and thus may not collide with the UAV. It should be noted that the preset ratio can be set by the staff according to actual needs, for example, the preset ratio can be 50%.

[0126] In some embodiments, based on the coordinate data of the obstacle in each frame of the front view image and the center point coordinate of the front view image, it is determined whether the obstacle moves towards the center point of the front view image, including: based on the coordinate data of the obstacle in each frame of the front view image, determining a plurality of displacement amounts of the obstacle, the displacement amount being the displacement amount of the obstacle between the current frame of the front view image and the next frame of the front view image; determining the moving speed of the obstacle based on each displacement amount; and determining whether the obstacle moves towards the center point of the front view image by using a preset formula and based on the moving speed of the obstacle and the center point coordinate of the front view image.

[0127] After obtaining the coordinate data of the obstacle in each frame of the front view image through the above target tracking process, the embodiments can first calculate the displacement amount of the center point of the obstacle from each frame to the next frame. The displacement amount of the center point of the obstacle can be calculated by the following formula:

[0128]

[0129] In the formula, is the displacement amount of the obstacle in the t frame of the front view image; (x t , y t ) is the coordinate data of the obstacle in the t frame of the front view image; (x t+1 , y t+1 ) is the coordinate data of the obstacle in the t+1 frame of the front view image; and N is the number of front view images.

[0130] Further, after obtaining the displacement amount of the obstacle in each frame of the front view image, the average displacement amount within a certain number of frames T (the number of frames T can be set according to requirements) can be calculated. Since the number of frames of the FPV camera is stable, the average displacement amount can be approximately equal to the moving speed of the obstacle within T frames.

[0131] In some embodiments, the moving speed of the obstacle is calculated by the following formula:

[0132]

[0133] wherein, is the moving speed of the obstacle; is the coordinate of the obstacle in the t frame front view image; is the coordinate of the obstacle in the t+1 frame front view image; and T is the preset total frame number of the front view image.

[0134] Further, whether the obstacle moves towards the center point of the front view image is determined by a preset formula. In some embodiments, the preset formula is:

[0135]

[0136] wherein, is the included angle between the moving speed and the center point vector; is the moving speed of the obstacle; is the center point vector; is the coordinate of the center point of the front view image; and a and c are the speeds in x and y directions respectively.

[0137] If , the obstacle is getting closer to the center; if , the obstacle is getting farther away from the center. According to the initial position of the trajectory, it can also be determined that the obstacle moves from left to right, from right to left, from top to bottom or from bottom to top.

[0138] It is worth mentioning that considering the overall flight logic of the unmanned aerial vehicle, it is basically divided into take-off, landing and cruising. Considering that the take-off stage usually takes place in a relatively open scene, the main collision occurs in the cruising and landing stages. In combination with the type of obstacle, it can be inferred that in the landing stage, all obstacles need to be detected, and for the cruising stage, considering that the unmanned aerial vehicle usually flies in the airspace above 50m, obstacles such as people, vehicles and power poles will not appear, so the results of target identification can be shielded, reducing the amount of calculation and false positives.

[0139] In the present embodiment, since most of the obstacles are close to the ground during the landing process, considering that the climbing rate of the unmanned aerial vehicle itself is relatively high and the response is more timely, in this stage, if an obstacle is detected to have a trend of moving towards the center of the image or the area ratio of the obstacle is large, a pull-up signal should be directly sent to the flight control chip, and the flight control chip should be executed to drive control.

[0140] In the UAV cruising phase, the task machine adopts an intelligent obstacle avoidance strategy based on motion trend analysis. Since the cruising area is usually relatively open, the core of the obstacle avoidance logic lies in analyzing the dynamic characteristics of the obstacles in the front view image. When it detects that the obstacles exhibit a motion trend of approaching the image center, the task machine will determine that there is a potential collision risk and trigger the active avoidance mechanism. Its obstacle avoidance logic is: reverse avoidance principle - if the obstacle moves from left to right, control the UAV to yaw left; otherwise, yaw right. For static obstacles, the task machine implements a mirror avoidance strategy according to the spatial position of the obstacles in the front view image: right avoidance when the obstacle appears on the left side of the picture, and left avoidance when the obstacle appears on the right side of the picture.

[0141] This two-way hierarchical obstacle avoidance strategy takes into account both the motion inertia of dynamic obstacles and the spatial relationship of static obstacles, and maps the position change in the image coordinate system to the heading adjustment amount of the UAV through kinematic inverse solution, while ensuring the effectiveness of obstacle avoidance and maximizing the continuity of the original route. Experimental data show that this strategy can achieve a successful obstacle avoidance rate of more than 92% within a 30-meter recognition distance, and the average heading correction amplitude required for safe avoidance is only 8-12 degrees.

[0142] Thus, by combining the improved ByteTrack algorithm, using low-confidence detection boxes to enhance the tracking continuity of occluded targets, and using Kalman filtering to predict the motion trend of obstacles, the accuracy of target tracking is improved; by adopting hierarchical obstacle avoidance decision-making, the obstacle avoidance strategy (such as reverse avoidance or emergency climb) is adaptively selected according to the flight phase (cruising / landing) and the type of obstacle (static / dynamic), and the response time is shortened to 86ms. Based on the YOLOv5 model accelerated by TensorRT and the ROS-MAVLink communication architecture, end-to-end low-latency processing is achieved. In addition, through the cooperative optimization of detection-tracking-decision based on time sequence perception, the obstacle avoidance robustness of the UAV in complex dynamic environments is significantly improved, and experiments show that it can still maintain an obstacle avoidance success rate of more than 95% under the conditions of wind speed 8m / s and light changes, solving the core problems of poor real-time performance, dynamic target missed detection, and rigid strategy in the prior art.

[0143] On the basis of the above-mentioned embodiments, the task machine of the UAV according to the embodiments of the present application comprises: Figure 5 is a structural schematic diagram of the task machine of the present application. As Figure 5 shown, the task machine 500 can include an acquisition module 510, a target identification module 520, a target tracking module 530, and an obstacle avoidance module 540.

[0144] The acquisition module 510 is configured to acquire the front-view image and the flight attitude data of the UAV in real time. The target identification module 520 is configured to perform target detection on each front-view image by using a pre-trained target identification model to obtain a target detection result. The target tracking module 530 is configured to, when the target detection result indicates that there is an obstacle, perform tracking on the obstacle by using a preset target tracking algorithm to obtain a tracking result of the obstacle. The obstacle avoidance module 540 is configured to determine whether the obstacle is a target obstacle based on the tracking result of the obstacle, the center point coordinate of the front-view image, and the area proportion of the obstacle in the front-view image, and determine an obstacle avoidance path based on the tracking result of the target obstacle and the flight attitude data when the obstacle is the target obstacle, so that the UAV performs obstacle avoidance according to the obstacle avoidance path.

[0145] In some embodiments, the tracking result of the obstacle includes coordinate data of the obstacle in each frame of front-view image; and the obstacle avoidance module 540 is specifically configured to determine whether the obstacle moves toward the center point of the front-view image based on the coordinate data of the obstacle in each frame of front-view image and the center point coordinate of the front-view image, and determine the obstacle as the target obstacle when it is determined that the obstacle moves toward the center point of the front-view image or the area proportion of the obstacle in the front-view image is greater than a preset threshold.

[0146] In some embodiments, the obstacle avoidance module 540 is further specifically configured to determine whether the obstacle moves toward the center point of the front-view image based on the coordinate data of the obstacle in each frame of front-view image and the center point coordinate of the front-view image, including: determining a plurality of displacement amounts of the obstacle based on the coordinate data of the obstacle in each frame of front-view image, the displacement amount being a displacement amount of the obstacle between a current frame of front-view image and a next frame of front-view image; determining a moving speed of the obstacle based on each displacement amount; and determining whether the obstacle moves toward the center point of the front-view image by using a preset formula and based on the moving speed of the obstacle and the center point coordinate of the front-view image.

[0147] In some embodiments, the preset formula is:

[0148]

[0149] In the formula, θ is an included angle between the moving speed vector and the center point vector; is a vector of the moving speed of the obstacle; is a center point vector; is a center point coordinate of the front-view image; and a and c are speeds in x and y directions respectively.

[0150] In some embodiments, the moving speed of the obstacle is calculated by the following formula:

[0151] In some embodiments, the tracking result of the obstacle includes coordinate data of the obstacle in each frame of front-view image; and the obstacle avoidance module 540 is specifically configured to determine whether the obstacle moves toward the center point of the front-view image based on the coordinate data of the obstacle in each frame of front-view image and the center point coordinate of the front-view image, and determine the obstacle as the target obstacle when it is determined that the obstacle moves toward the center point of the front-view image or the area proportion of the obstacle in the front-view image is greater than a preset threshold.

[0146] In some embodiments, the obstacle avoidance module 540 is further specifically configured to determine whether the obstacle moves toward the center point of the front-view image based on the coordinate data of the obstacle in each frame of front-view image and the center point coordinate of the front-view image, including: determining a plurality of displacement amounts of the obstacle based on the coordinate data of the obstacle in each frame of front-view image, the displacement amount being a displacement amount of the obstacle between a current frame of front-view image and a next frame of front-view image; determining a moving speed of the obstacle based on each displacement amount; and determining whether the obstacle moves toward the center point of the front-view image by using a preset formula and based on the moving speed of the obstacle and the center point coordinate of the front-view image.

[0147] In some embodiments, the preset formula is:

[0148]

[0149] In the formula, θ is an included angle between the moving speed vector and the center point vector; is a vector of the moving speed of the obstacle; is a center point vector; is a center point coordinate of the front-view image; and a and c are speeds in x and y directions respectively.

[0150] In some embodiments, the moving speed of the obstacle is calculated by the following formula:

[0151]

[0152] wherein, is a moving speed of the obstacle; is a coordinate of the obstacle in a t frame front view image; is a coordinate of the obstacle in a t+1 frame front view image; T is a preset total frame number of the front view image.

[0153] It should be noted that details not disclosed in the task machine 500 of the embodiment are referred to the details disclosed in the embodiment of the obstacle avoidance method of the unmanned aerial vehicle in the specification, which will not be described here.

[0154] On the basis of the above embodiment, the application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the obstacle avoidance method of the unmanned aerial vehicle provided by the above method, and the method comprises: acquiring a front view image and flight attitude data of the unmanned aerial vehicle in real time, and using a pre-trained target recognition model to perform target detection on each front view image to obtain a target detection result; in the case that the target detection result is that there is an obstacle, using a preset target tracking algorithm to track the obstacle to obtain a tracking result of the obstacle; based on the tracking result of the obstacle, a center point coordinate of the front view image and an area proportion of the obstacle in the front view image, judging whether the obstacle is a target obstacle, and in the case that the obstacle is the target obstacle, determining an obstacle avoidance path based on the tracking result of the target obstacle and the flight attitude data, so that the unmanned aerial vehicle avoids obstacles according to the obstacle avoidance path.

[0155] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. Those skilled in the art can understand and implement without creative labor.

[0156] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary general hardware platform, and of course, it can also be realized by hardware. Based on such understanding, the above technical solutions essentially or say the part of the prior art which makes contributions can be embodied in the form of software product, which can be stored in a computer readable storage medium such as ROM / RAM, magnetic disk, optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the method described in each embodiment or some part of the embodiment.

Claims

1. An obstacle avoidance method for unmanned aerial vehicles (UAVs), characterized in that, include: The forward view image and flight attitude data of the UAV are acquired in real time, and the target detection is performed on each forward view image using a pre-trained target recognition model to obtain the target detection results; If the target detection result indicates the presence of an obstacle, a preset target tracking algorithm is used to track the obstacle to obtain the tracking result of the obstacle; Based on the tracking results of the obstacle, the coordinates of the center point of the front view image, and the area ratio of the obstacle in the front view image, it is determined whether the obstacle is a target obstacle. If the obstacle is the target obstacle, an obstacle avoidance path is determined based on the tracking results of the target obstacle and the flight attitude data, so that the UAV can avoid obstacles according to the obstacle avoidance path. The obstacle tracking result includes the coordinate data of the obstacle in each frame of the front view image; the step of determining whether the obstacle is a target obstacle based on the obstacle tracking result, the center point coordinates of the front view image, and the area ratio of the obstacle in the front view image includes: Based on the coordinate data of the obstacle in each frame of the front view image and the coordinates of the center point of the front view image, it is determined whether the obstacle is moving towards the center point of the front view image; If it is determined that the obstacle is moving toward the center point of the front view image or the area of ​​the obstacle in the front view image is greater than a preset percentage, then the obstacle is designated as the target obstacle.

2. The obstacle avoidance method for a drone according to claim 1, characterized in that, The step of determining whether the obstacle is moving toward the center point of the front view image based on the coordinate data of the obstacle in each frame of the front view image and the coordinates of the center point of the front view image includes: Based on the coordinate data of the obstacle in the front view image of each frame, multiple displacements of the obstacle are determined, and the displacements are the displacements of the obstacle between the front view image of the current frame and the front view image of the next frame. The moving speed of the obstacle is determined based on each of the displacements; Using a preset formula and based on the obstacle's moving speed and the center point coordinates of the front view image, it is determined whether the obstacle is moving towards the center point of the front view image.

3. The obstacle avoidance method for a drone according to claim 2, characterized in that, The preset formula is: In the formula, The angle between the moving speed and the center point vector; The moving speed of the obstacle; The center point vector; Let a be the coordinates of the center point of the front view image; and let a and c be the velocities in the x and y directions, respectively.

4. The obstacle avoidance method for a drone according to claim 2, characterized in that, The moving speed of the obstacle is calculated using the following formula: In the formula, The moving speed of the obstacle; Let the coordinates of the obstacle be the coordinates of the foreground image in frame t; The coordinates of the obstacle in the front view image at frame t+1; T is the preset total number of frames for the front view image.

5. The obstacle avoidance method for a UAV according to any one of claims 1 to 4, characterized in that, The training steps of the target recognition model include: A training dataset is obtained, which includes multiple sample images with bounding boxes and annotation information, and each sample image is processed using Mosaic data augmentation and adaptive image scaling. Based on the bounding boxes in each of the sample images and using a preset adaptive anchor box algorithm, the target anchor box is obtained; The processed sample images are input into a pre-constructed target recognition model. The target recognition model obtains the target recognition result based on the target anchor box, and calculates the loss value using the CIOU loss function based on the target recognition result and the annotation information. The parameters of the target recognition model are updated based on the loss value until the target recognition model converges, thus obtaining the trained target recognition model.

6. The obstacle avoidance method for a drone according to claim 5, characterized in that, The step of obtaining the target anchor box based on the bounding boxes in each of the sample images and using a preset adaptive anchor box algorithm includes: If the recall rate of the initial anchor box of the target recognition model is lower than a preset threshold, N anchor boxes are randomly initialized, where N is a positive integer; The anchor boxes are updated using a genetic algorithm, a clustering algorithm, the bounding boxes in each of the sample images, and the anchor boxes until each anchor box meets a preset condition. Then, each anchor box is used as the target anchor box. The preset condition is that the recall rate of each anchor box is higher than the preset threshold.

7. A mission controller for an unmanned aerial vehicle (UAV), characterized in that, include: The acquisition module is used to acquire the forward view image and flight attitude data of the UAV in real time; The target recognition module is used to perform target detection on each of the front view images using a pre-trained target recognition model to obtain target detection results; The target tracking module is used to track the obstacle using a preset target tracking algorithm when the target detection result indicates the presence of an obstacle, and obtain the tracking result of the obstacle; The obstacle avoidance module is used to determine whether the obstacle is a target obstacle based on the tracking result of the obstacle, the coordinates of the center point of the front view image, and the area ratio of the obstacle in the front view image. If the obstacle is the target obstacle, the module determines an obstacle avoidance path based on the tracking result of the target obstacle and the flight attitude data, so that the UAV can avoid obstacles according to the obstacle avoidance path. The obstacle tracking result includes the coordinate data of the obstacle in each frame of the front view image; the obstacle avoidance module is specifically used to: determine whether the obstacle is moving towards the center point of the front view image based on the coordinate data of the obstacle in each frame of the front view image and the coordinates of the center point of the front view image; if it is determined that the obstacle is moving towards the center point of the front view image or the area of ​​the obstacle in the front view image is greater than a preset proportion, then the obstacle is regarded as the target obstacle.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the obstacle avoidance method for the UAV as described in any one of claims 1-6.

9. An unmanned aerial vehicle (UAV) system, characterized in that, Includes the mission aircraft, camera, and flight control chip as described in claim 7, wherein the mission aircraft, the camera, and the flight control chip utilize ROS1 to achieve data interaction; The camera is used to acquire a frontal view image of the UAV in real time and send the frontal view image to the mission unit; The flight control chip is used to acquire the flight attitude data of the UAV and send the flight attitude data to the mission machine; The mission unit is used to receive forward-view images and flight attitude data of the UAV in real time, and to perform target detection on each forward-view image using a pre-trained target recognition model to obtain target detection results. If the target detection results indicate the presence of an obstacle, a preset target tracking algorithm is used to track the obstacle to obtain the obstacle tracking result. Based on the obstacle tracking result, the center point coordinates of the forward-view image, and the area ratio of the obstacle in the forward-view image, it is determined whether the obstacle is a target obstacle. If the obstacle is the target obstacle, an obstacle avoidance path is determined based on the target obstacle tracking result and the flight attitude data, so that the UAV can avoid obstacles according to the obstacle avoidance path. The obstacle tracking result includes the coordinate data of the obstacle in each frame of the forward-view image. Specifically, the mission unit is used to: determine whether the obstacle is moving towards the center point of the forward-view image based on the coordinate data of the obstacle in each frame of the forward-view image and the center point coordinates of the forward-view image; if it is determined that the obstacle is moving towards the center point of the forward-view image or the area ratio of the obstacle in the forward-view image is greater than a preset ratio, then the obstacle is designated as the target obstacle.

Images