Unmanned aerial vehicle multi-mode automatic obstacle avoidance method based on Beidou system

Abstract

The invention discloses an unmanned aerial vehicle multi-mode automatic obstacle avoidance method based on a Beidou system, and relates to the navigation and obstacle avoidance field in the robot field. Comprising unmanned aerial vehicle starting and navigation, image processing and feature extraction, environment mapping and path planning, obstacle recognition and avoidance, multi-modal fusion dynamic obstacle avoidance, flight control and information transmission and an emergency obstacle avoidance mechanism in an emergency. According to the invention, a Beidou system, machine vision, a deep learning algorithm and multi-sensor fusion are combined, high-precision positioning, real-time path optimization and dynamic obstacle avoidance are realized, and safe and efficient flight of the unmanned aerial vehicle in a complex environment is ensured. Through the three-dimensional mapping and ant colony optimization algorithm, the path planning efficiency is improved, the method is adaptive to the environmental change, and the autonomy, flexibility and task execution capability rate of the unmanned aerial vehicle are significantly enhanced.

Description

Technical Field

[0001] The present invention relates to the field of navigation and obstacle avoidance in the field of robots, and in particular to a multi-modal automatic obstacle avoidance method for unmanned aerial vehicles based on the Beidou system. Background Art

[0002] With the development of drone technology, its application is becoming more and more extensive. In the agricultural field, drones can be used for automatic inspections and spraying of pesticides. With the help of Beidou RTK positioning technology, high-precision positioning of 1-3 cm can be achieved to ensure uniform spraying and reduce human errors. In logistics and transportation, drones need to have obstacle avoidance capabilities, especially in complex urban environments. The logistics pilot in Shenzhen, Guangdong, uses a combination of Beidou, GPS, visual sensors and lidar to achieve high-precision positioning, helping drones complete delivery tasks in areas with dense high-rise buildings and unstable signals. In disaster relief, Beidou drones combine thermal imaging and high-definition cameras to efficiently survey disaster areas, quickly locate trapped people, and improve rescue efficiency. In the field of security, during patrol missions in Shanghai's commercial districts, drones use the Beidou system to fly autonomously, avoid obstacles, and transmit videos in real time to ensure safety.

[0003] Although the prior art has adopted the Beidou system, GPS, laser radar and visual sensors in the field of unmanned aerial vehicle obstacle avoidance, it still has problems such as unstable positioning accuracy, insufficient obstacle avoidance capability, delayed real-time response, and insufficient path planning optimization in complex environments (such as cities, high-rise dense areas and bad weather). In addition, the existing system lacks sufficient adaptive capabilities and is difficult to deal with dynamic obstacles and emergencies, resulting in hidden dangers in flight safety and efficiency. Therefore, the present invention provides a multi-modal automatic obstacle avoidance method for unmanned aerial vehicles based on the Beidou system to solve the problems raised in the above background technology. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a multi-modal automatic obstacle avoidance method for an unmanned aerial vehicle based on the Beidou system to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A multi-modal automatic obstacle avoidance method for a UAV based on the Beidou system comprises the following steps:

[0007] S1. Drone startup and navigation: Start the drone, use the Beidou system to perform preliminary positioning to determine the current location, and autonomously navigate to the preset operation area; at the same time, the drone uses machine vision equipment to obtain image data of the set area, including environmental characteristics, obstacles and path information;

[0008] S2, image processing and feature extraction: The acquired image is preliminarily processed through feature extraction, data association and pose estimation at the front end, and then optimized using the maximum a posteriori probability estimation at the back end;

[0009] S3, Environmental Mapping and Path Planning: Construct a 3D grid map model based on the processed image information, and perform preliminary route planning. Use the ant colony optimization algorithm to adjust the transfer probability and pheromone distribution to optimize and generate the optimal flight path.

[0010] S4, Obstacle Identification and Avoidance: Based on the deep learning target detection model, the image features are extracted and the training model is matched to accurately identify the location and category of obstacles;

[0011] S5, Dynamic obstacle avoidance with multi-modal fusion: Based on the identified obstacle location and category, combined with LiDAR and ultrasonic radar sensors, the environment is scanned in real time to detect the location and distance of obstacles, and the flight path is dynamically adjusted through multi-sensor data fusion to achieve accurate obstacle avoidance;

[0012] S6, Flight control and information transmission: The flight attitude is controlled by the flight control system, and the sensors are used to make real-time adjustments. The obstacle recognition results and flight images are transmitted to the ground station in real time through the NRF wireless transmission module, which facilitates real-time intervention by monitoring personnel.

[0013] As a further solution of the present invention, the specific method steps of step S2 are as follows:

[0014] S21, extracting feature points from the image and establishing matching relationships in multiple frames of images, accurately optimizing the camera pose by minimizing the reprojection error to accurately determine the spatial position and orientation of the camera;

[0015] S22. Use visual odometer to estimate the change of the camera pose of the drone, combine loop detection to correct the map error, and globally optimize the pose and map points based on the 3D map points and loop information to improve the accuracy of environment mapping.

[0016] S23. Extended Kalman filtering is used to filter the sensor noise in the mapping process, and the maximum a posteriori probability estimation is combined to optimize the pose and map points, and the estimation accuracy and stability are improved by fusing prior information and observation data.

[0017] As a further solution of the present invention, the specific method steps of step S3 are as follows:

[0018] S31, constructing a three-dimensional grid map based on the processed image information, dividing the environment into grid units, and marking the positions of obstacles and free spaces;

[0019] S32, using the grid map model and the target location, using a path planning algorithm to calculate the initial route of the UAV and determine a feasible path from the starting point to the target;

[0020] S33, using ant colony optimization algorithm, by simulating the behavior of ants searching for paths, selecting the optimal path according to pheromone concentration and transfer probability, and updating pheromones in each iteration to gradually optimize the flight path;

[0021] S34. Output the optimal flight path optimized by the ant colony optimization algorithm, and dynamically adjust the path based on real-time environmental feedback to ensure safe, smooth and efficient flight.

[0022] As a further solution of the present invention, the specific method steps of the ant colony optimization algorithm in step S33 are as follows:

[0023] S331, constructing a grid map model, randomly setting obstacles, and initializing various parameters of the ant colony optimization algorithm;

[0024] S332, initialize ant colony information, set the current node as the starting position, calculate the Euclidean distance between nodes, and initialize pheromone distribution and related data table items;

[0025] S333, each ant calculates the transition probability of each node in the allowed table according to the improved transition probability function, selects the next node by the roulette method, adds its ID to the path table, and calculates and updates the path length;

[0026] S334, determine whether the target node is in the taboo table, if so, increase the current iteration number and return to step S333 until all ants complete the search;

[0027] S335, comparing the length of each path obtained in the current cycle, recording the best path number in the cycle and the obtained length and related path information, and adjusting the pheromone according to the pheromone volatilization and update rules;

[0028] S336. Determine whether the current number of iterations has reached the maximum value. If not, reset the taboo table, path length and node table, increase the number of iterations and return to step S333 for a new round of search; if the maximum number of iterations has been reached, end the algorithm and output the best path.

[0029] As a further solution of the present invention, the main process of feature recognition of obstacle objects in step S4 is as follows:

[0030] S41, preprocessing each input image, scaling the image to the network input size and normalizing it through adaptive anchor frame calculation and adaptive scaling method;

[0031] S42, using the CSPDarknet53 structure pre-trained on a large dataset and the Focus structure as the benchmark network for feature extraction;

[0032] S43, add network layers to collect feature maps at different stages;

[0033] S44, the target detection model structure based on deep learning, the model processes the features of each network layer, and finally outputs the location, category and confidence score of the obstacle object to complete the identification and positioning of the obstacle.

[0034] As a further solution of the present invention, the specific method steps of step S5 are as follows:

[0035] S51, scan the environment in real time through laser radar and ultrasonic radar, detect the location and distance of obstacles, and classify and identify their types;

[0036] S52. Using the data collected by the sensor, identify the location of the obstacle, and determine the type of the obstacle by combining deep learning or traditional image processing methods;

[0037] S53, combining the different detection ranges of laser radar and ultrasonic radar, to conduct all-round obstacle detection and update environmental information in real time;

[0038] S54, generating an obstacle avoidance strategy according to the location and type of the obstacle, and adjusting the flight path, speed or altitude to avoid collision;

[0039] S55, combining the obstacle detection results and the flight control system, automatically adjusting the flight path and speed to optimize the flight route.

[0040] As a further solution of the present invention, the following steps are further included:

[0041] S7. Emergency obstacle avoidance mechanism in emergency situations: When it is detected that the flight path cannot completely avoid obstacles, the emergency avoidance mode is automatically triggered in combination with the precise positioning data and terrain information provided by the Beidou system, and the flight altitude is adjusted or the backup route is switched to ensure the flight safety of the UAV.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. The present invention combines the Beidou system, machine vision and deep learning algorithms, and the drone can achieve high-precision positioning, and accurately identify and process obstacle information in the environment through visual sensors. The system can ensure that the drone accurately navigates and smoothly reaches the preset operation area in complex environments, such as high-rise dense areas or farmland, reducing human intervention and operation errors.

[0044] 2. The present invention constructs a three-dimensional grid map and uses the ant colony optimization (ACO) algorithm for path planning. The system can respond to environmental changes in real time and dynamically adjust the flight path. The ACO algorithm optimizes the path based on real-time feedback in each iteration, which can effectively avoid obstacles and reduce flight time, ensuring efficient and safe flight of drones in complex or crowded environments.

[0045] 3. The present invention combines the multi-dimensional information of laser radar, ultrasonic radar and visual sensors. The system can fully monitor obstacles at different flight altitudes and angles, ensuring that the drone can detect obstacles in all directions and at all times. This multi-sensor data fusion improves the system's recognition accuracy and response speed for various obstacles, effectively ensuring flight safety.

[0046] 4. The present invention can autonomously learn and adapt to obstacle characteristics in different environments through the combination of deep learning and traditional image processing technology. The system can automatically adjust obstacle recognition and avoidance strategies according to the changing flight environment and mission requirements, improve flexibility and adaptability in complex environments, and further enhance the autonomous flight capability and mission execution efficiency of the UAV. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of the method steps of the present invention;

[0048] Figure 2 Schematic diagram of the steps of the ant colony optimization (ACO) algorithm of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the implementation cases described are only part of the implementation content of the present invention, not all. Based on the implementation content of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0050] Embodiment 1:

[0051] See also Figures 1-2 In an embodiment of the present invention, a multi-modal automatic obstacle avoidance method for a UAV based on the Beidou system comprises the following steps:

[0052] S1. Drone startup and navigation: Start the drone, perform preliminary positioning through the Beidou system, determine the current position, and autonomously navigate to the preset operation area; at the same time, the drone uses machine vision equipment to obtain image data of the set area, including environmental characteristics, obstacles and path information, to provide data support for subsequent processing.

[0053] S2, image processing and feature extraction: The acquired image is preliminarily processed through feature extraction, data association and pose estimation at the front end, and then optimized using the maximum a posteriori probability estimation at the back end;

[0054] S3, Environmental Mapping and Path Planning: Construct a 3D grid map model based on the processed image information, and perform preliminary route planning. Use the ant colony optimization algorithm to adjust the transfer probability and pheromone distribution to optimize and generate the optimal flight path.

[0055] S4, Obstacle Identification and Avoidance: Based on the deep learning target detection model, the image features are extracted and the training model is matched to accurately identify the location and category of obstacles;

[0056] S5, Dynamic obstacle avoidance with multi-modal fusion: Based on the identified obstacle location and category, combined with LiDAR and ultrasonic radar sensors, the environment is scanned in real time to detect the location and distance of obstacles, and the flight path is dynamically adjusted through multi-sensor data fusion to achieve accurate obstacle avoidance;

[0057] S6, Flight control and information transmission: Control the flight attitude through the flight control system, use sensors to adjust in real time, and transmit obstacle recognition results and flight images to the ground station in real time through the NRF wireless transmission module, so that monitoring personnel can intervene in real time;

[0058] S7. Emergency obstacle avoidance mechanism in emergency situations: When it is detected that the flight path cannot completely avoid obstacles, the emergency avoidance mode is automatically triggered in combination with the precise positioning data and terrain information provided by the Beidou system, and the flight altitude is adjusted or the backup route is switched to ensure the flight safety of the UAV.

[0059] Specifically, the specific method steps of step S2 are as follows:

[0060] S21. Feature point extraction and camera pose optimization: Extract feature points from the image and establish matching relationships in multiple frames, and optimize the camera pose by minimizing the reprojection error. The reprojection error refers to the error in projecting the 3D world coordinate point into the 2D image coordinate system through the camera model. Optimizing this error can improve the accuracy of the camera pose (position and orientation) estimation, thereby accurately determining the spatial position and orientation of the camera.

[0061] S22. Visual odometry and loop closure detection: Estimate the change in the camera pose of the drone through visual odometry, which estimates the position and pose of the drone by analyzing motion changes in continuous image frames. Combined with loop closure detection, map errors can be corrected. Loop closure detection is when the drone passes through an area that has been explored again during flight. The system can recognize this situation and make position corrections to reduce cumulative errors. Based on three-dimensional map points and loop closure information, poses and map points are globally optimized. Loop closure information can provide global consistency and enhance the accuracy of the map.

[0062] S23. Extended Kalman Filter and Maximum A Posteriori Probability Estimation: Extended Kalman Filter (EKF) is used to filter sensor noise during the mapping process. Extended Kalman Filter is a nonlinear filtering method that can reduce the impact of sensor noise in dynamic environments and improve the stability and accuracy of estimates. Combined with Maximum A Posteriori Probability Estimation (MAP) to optimize pose and map points. Maximum A Posteriori Probability Estimation optimizes pose and map points by fusing prior information and observation data to further improve estimation accuracy. Through this fusion method, uncertainty can be eliminated while maintaining accuracy, thereby improving the stability and reliability of environmental mapping.

[0063] Specifically, the specific method steps of step S3 are as follows:

[0064] S31. Construct a three-dimensional grid map model: Based on the processed image information, construct a three-dimensional grid map model, dividing the environment into a three-dimensional grid grid in one step, where each grid represents a certain spatial area. Each grid will be marked as "obstacle" or "free space", where obstacles are inaccessible areas and free space is an area where you can fly. In this way, a clear environmental structure can be provided for subsequent path planning. In this process, obstacle information extracted using machine vision and deep learning models is used to fill in obstacle areas to ensure the accuracy of the map and to reflect environmental changes around the drone in real time.

[0065] S32. Preliminary route calculation and path planning: The path planning algorithm is used to calculate the preliminary route of the drone through the grid map model and the target position. This algorithm calculates feasible paths from the starting point to the target based on the obstacles and target position of the environment. These paths will take into account the existence of obstacles and ensure that the path is in free space. Usually, the path planning algorithm uses classic graph search algorithms such as the A* algorithm and the Dijkstra algorithm to select the shortest or optimal path.

[0066] S33. Optimize the path using the Ant Colony Optimization (ACO) algorithm: Use the Ant Colony Optimization (ACO) algorithm to further optimize the path. The Ant Colony Optimization algorithm simulates the behavior of ants in finding food paths and selects paths through the propagation and concentration changes of pheromones. Each ant represents a candidate path, and in each round of iteration, the optimal path is selected based on the pheromone concentration and the probability of transition. In each iteration, the ants choose the path to move forward based on probability. Paths with high pheromone concentrations will be given priority, and the pheromone concentration will be updated on the passed paths, thereby gradually guiding the ant colony to concentrate on the optimal path. After each round of iteration, the pheromone will evaporate, thereby avoiding falling into the local optimal solution, and finally finding the global optimal or near-optimal flight path.

[0067] S34, Optimized Path Output: Output the optimal flight path optimized by the Ant Colony Optimization (ACO) algorithm, and adjust the path in combination with real-time environmental feedback to ensure safe, smooth and efficient flight. During the flight of the drone, the system obtains real-time environmental feedback, monitors the changes in obstacles and the real-time status of the flight path, and adjusts the flight path based on real-time environmental data. Real-time adjustment of the path ensures that the drone can cope with sudden obstacles in a dynamic environment and continue to perform its mission without collision.

[0068] Specifically, the specific method steps of the ant colony optimization algorithm in step S33 are as follows:

[0069] S331. Build a grid map model, randomly set obstacles, and initialize various parameters in the ant colony optimization algorithm, including the total number of nodes CityNum, the number of ant iterations IterNum, the initialization pheromone amount p, the track area safety factor weight w, etc.

[0070] S332, initialize ant colony information, set the current node as the starting position, calculate the Euclidean distance between nodes, and initialize pheromone distribution and each table item data;

[0071] S333, each ant k (k = 0, 1, 2, ... AntNum) calculates the transition probability of each node in the allowed table according to the improved transition probability function, selects the next node by the roulette method, adds its ID to the path table, and calculates and updates the path length;

[0072] S334, determine whether the target node is in the taboo table, if so, increase the current iteration number and return to step S333 until all ants have completed the search;

[0073] S335, comparing the length of each path obtained in this cycle, recording the best path number in this cycle and the obtained length and related path information, and volatilizing and updating according to the pheromone update rule;

[0074] S336. Determine whether the current number of iterations has reached the maximum value. If not, reset the taboo table, path length and node table, increase the number of iterations and return to step S333 for a new round of search; if the maximum number of iterations has been reached, end the algorithm and output the best path.

[0075] Specifically, the main process of feature recognition of obstacle objects in step S4 is as follows:

[0076] S41, Image preprocessing: Preprocess each input image. This includes scaling the image through adaptive anchor box calculation and adaptive scaling methods to make it meet the input size requirements of the neural network. After the image is scaled, it is also necessary to perform normalization to map the pixel values ​​to a standard range (such as between 0 and 1) to ensure the uniformity of the input data and the efficiency of the processing;

[0077] S42. Feature extraction using pre-trained networks: Use the CSPDarknet53 (CSP-Darknet53) structure pre-trained on a large dataset for feature extraction. CSPDarknet53 is an efficient neural network structure in the YOLOv4 series that can efficiently extract features from images. It is based on the Focus structure as the baseline network. The Focus module can effectively enhance the network's feature extraction capabilities by dividing the input image into multiple small blocks and performing feature fusion, especially for complex backgrounds and small target objects;

[0078] S43. Feature Collection: Network Layer Addition and Multi-stage Feature Map Collection Additional network layers are added to the network to extract and collect feature maps of different levels from different stages. The purpose of this is to enhance the model's multi-scale feature perception capabilities, especially when the target objects vary greatly in size and shape. These feature maps help identify various types of obstacles, not only limited to large foreground objects, but also effectively capture smaller, detailed obstacle information;

[0079] S44. Object detection and positioning based on deep learning: The object detection model based on deep learning can accurately determine the location, category and confidence score of the obstacle by processing the feature map output by each layer of the network. The structure of the model includes: input, backbone network, neck part, head and output. In this structure, the backbone network is responsible for extracting basic features, the neck part extracts more complex multi-scale features, and the head predicts and outputs the final result based on these features. Specifically, the position refers to the bounding box of the obstacle in the image, that is, the location information of the object; the category represents the type of obstacle recognized by the model (such as trees, buildings, vehicles, etc.); the confidence score reflects the reliability or accuracy of the model's prediction results.

[0080] Through this complete processing flow, the target detection model can accurately identify and locate obstacles and provide reliable information for subsequent obstacle avoidance decisions.

[0081] Specifically, the specific method steps of step S5 are as follows:

[0082] S51, Environmental scanning: Scan the surrounding environment in real time through laser radar and ultrasonic radar, detect and record the location and distance of obstacles. According to the distance information fed back by the sensor, judge the distance and relative position of the obstacle. At the same time, preliminarily classify the obstacles and identify their types (such as static obstacles or dynamic obstacles);

[0083] S52, Multi-state data fusion: Use data collected from LiDAR, ultrasonic radar and visual sensors to accurately identify the location of obstacles. Combined with deep learning or traditional image processing methods, further analyze and determine the specific types of obstacles, such as trees, buildings, vehicles, etc.

[0084] S53, all-round detection: Combine the different detection ranges of LiDAR and ultrasonic radar to conduct all-round and multi-angle obstacle detection to ensure real-time monitoring of obstacles in all directions. As the flight progresses, the sensor continuously obtains updated environmental data to ensure that the location information of obstacles is always up to date;

[0085] S54, obstacle avoidance strategy generation: Generate a corresponding obstacle avoidance strategy based on the location and type of the detected obstacle. Adjust the flight path, flight speed or flight altitude according to real-time environmental changes to avoid collision with obstacles;

[0086] S55, Path Optimization: Combine obstacle detection results with the flight control system to automatically adjust the flight path and flight speed and optimize route planning, ensuring that the drone can fly safely and smoothly in a dynamic environment while avoiding obstacles and completing mission objectives.

[0087] It should be pointed out that the basic autonomous flight workflow that is highly related to the flight control (including attitude, position, and altitude control) and obstacle identification and avoidance steps of the above-mentioned Beidou system-based multi-modal automatic obstacle avoidance method for unmanned aerial vehicles is as follows: The basic flight control layer includes attitude PID control, altitude PID control, and horizontal position PID control. The specific process is as follows:

[0088] Attitude control: After multi-sensor information fusion and Kalman filtering based on the AHRS (attitude and heading reference system) of the MEMS_IMU motion sensor, the heading attitude angle (Pitch, Roll, Yaw) can be accurately output. The attitude PID controller uses the calculated heading attitude angle as the feedback quantity and the horizontal stable attitude quantity as the expected quantity, thereby achieving stable attitude control of the drone.

[0089] Altitude control: Altitude PID control uses the satellite navigation altitude data obtained by the Beidou receiver as feedback, while the desired altitude control comes from the host computer, remote controller or preset target altitude. This control layer ensures that the drone can maintain a stable absolute altitude.

[0090] Horizontal position control: The horizontal position PID controller uses the satellite navigation latitude and longitude coordinate data received by the Beidou receiver as feedback and the UAV's horizontal motion target (desired control quantity) as control input to achieve precise position control of the UAV, which is especially important when hovering at high altitudes.

[0091] Integrated Navigation: The entire flight control system uses INS (Inertial Navigation System) and BDS (Beidou Satellite Navigation System) combined navigation to ensure that the UAV can fly stably during high-altitude cruising operations.

[0092] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A multi-modal automatic obstacle avoidance method for unmanned aerial vehicles based on the Beidou system, characterized in that: The following steps are involved: S1. Drone startup and navigation: Start the drone, use the Beidou system to perform preliminary positioning to determine the current location, and autonomously navigate to the preset operation area; at the same time, the drone uses machine vision equipment to obtain image data of the set area, including environmental characteristics, obstacles and path information; S2, image processing and feature extraction: The acquired image is preliminarily processed through feature extraction, data association and pose estimation at the front end, and then optimized using the maximum a posteriori probability estimation at the back end; S3, Environmental Mapping and Path Planning: Construct a 3D grid map model based on the processed image information, and perform preliminary route planning. Use the ant colony optimization algorithm to adjust the transfer probability and pheromone distribution to optimize and generate the optimal flight path. S4, Obstacle Identification and Avoidance: Based on the deep learning target detection model, the image features are extracted and the training model is matched to accurately identify the location and category of obstacles; S5, Dynamic obstacle avoidance with multi-modal fusion: Based on the identified obstacle location and category, combined with LiDAR and ultrasonic radar sensors, the environment is scanned in real time to detect the location and distance of obstacles, and the flight path is dynamically adjusted through multi-sensor data fusion to achieve accurate obstacle avoidance; S6, Flight control and information transmission: The flight attitude is controlled by the flight control system, and the sensors are used to make real-time adjustments. The obstacle recognition results and flight images are transmitted to the ground station in real time through the NRF wireless transmission module, which facilitates real-time intervention by monitoring personnel.

2. The multi-modal automatic obstacle avoidance method for unmanned aerial vehicles based on the Beidou system according to claim 1 is characterized in that: The specific method steps of step S2 are as follows: S21, extracting feature points from the image and establishing matching relationships in multiple frames of images, accurately optimizing the camera pose by minimizing the reprojection error to accurately determine the spatial position and orientation of the camera; S22. Use visual odometer to estimate the change of the camera pose of the drone, combine loop detection to correct the map error, and globally optimize the pose and map points based on the 3D map points and loop information to improve the accuracy of environment mapping. S23. Extended Kalman filtering is used to filter the sensor noise in the mapping process, and the maximum a posteriori probability estimation is combined to optimize the pose and map points, and the estimation accuracy and stability are improved by fusing prior information and observation data.

3. The multi-modal automatic obstacle avoidance method for unmanned aerial vehicles based on the Beidou system according to claim 1 is characterized in that: The specific method steps of step S3 are as follows: S31, constructing a three-dimensional grid map based on the processed image information, dividing the environment into grid cells, and marking the positions of obstacles and free spaces; S32, using the grid map model and the target location, using a path planning algorithm to calculate the initial route of the UAV and determine a feasible path from the starting point to the target; S33, using ant colony optimization algorithm, by simulating the behavior of ants searching for paths, selecting the optimal path according to pheromone concentration and transfer probability, and updating pheromones in each iteration to gradually optimize the flight path; S34. Output the optimal flight path optimized by the ant colony optimization algorithm, and dynamically adjust the path based on real-time environmental feedback to ensure safe, smooth and efficient flight.

4. The multi-modal automatic obstacle avoidance method for unmanned aerial vehicles based on the Beidou system according to claim 3 is characterized in that: The specific method steps of the ant colony optimization algorithm in step S33 are as follows: S331, constructing a grid map model, randomly setting obstacles, and initializing various parameters of the ant colony optimization algorithm; S332, initialize ant colony information, set the current node as the starting position, calculate the Euclidean distance between nodes, and initialize pheromone distribution and related data table items; S333, each ant calculates the transition probability of each node in the allowed table according to the improved transition probability function, selects the next node by the roulette method, adds its ID to the path table, and calculates and updates the path length; S334, determine whether the target node is in the taboo table, if so, increase the current iteration number and return to step S333 until all ants complete the search; S335, comparing the length of each path obtained in the current cycle, recording the best path number in the cycle and the obtained length and related path information, and adjusting the pheromone according to the pheromone volatilization and update rules; S336. Determine whether the current number of iterations has reached the maximum value. If not, reset the taboo table, path length and node table, increase the number of iterations and return to step S333 for a new round of search; if the maximum number of iterations has been reached, end the algorithm and output the best path.

5. The multi-modal automatic obstacle avoidance method for unmanned aerial vehicles based on the Beidou system according to claim 1 is characterized in that: The main process of feature recognition of obstacle objects in step S4 is as follows: S41, preprocessing each input image, scaling the image to the network input size and normalizing it through adaptive anchor frame calculation and adaptive scaling method; S42, using the CSPDarknet53 structure pre-trained on a large dataset and the Focus structure as the benchmark network for feature extraction; S43, add network layers to collect feature maps at different stages; S44, the target detection model structure based on deep learning, the model processes the features of each network layer, and finally outputs the location, category and confidence score of the obstacle object to complete the identification and positioning of the obstacle.

6. The multi-modal automatic obstacle avoidance method for unmanned aerial vehicles based on the Beidou system according to claim 1 is characterized in that: The specific method steps of step S5 are as follows: S51, scan the environment in real time through laser radar and ultrasonic radar, detect the location and distance of obstacles, and classify and identify their types; S52. Using the data collected by the sensor, identify the location of the obstacle, and determine the type of the obstacle by combining deep learning or traditional image processing methods; S53, combining the different detection ranges of laser radar and ultrasonic radar, to conduct all-round obstacle detection and update environmental information in real time; S54, generating an obstacle avoidance strategy according to the location and type of the obstacle, and adjusting the flight path, speed or altitude to avoid collision; S55, combining the obstacle detection results and the flight control system, automatically adjusting the flight path and speed to optimize the flight route.

7. The multi-modal automatic obstacle avoidance method for unmanned aerial vehicles based on the Beidou system according to claim 1 is characterized in that: Further comprising the steps of: S7. Emergency obstacle avoidance mechanism in emergency situations: When it is detected that the flight path cannot completely avoid obstacles, the emergency avoidance mode is automatically triggered in combination with the precise positioning data and terrain information provided by the Beidou system, and the flight altitude is adjusted or the backup route is switched to ensure the flight safety of the UAV.

Images