Intelligent unmanned aerial vehicle navigation method and system based on bimodal obstacle feature extraction

Abstract

The invention belongs to the technical field of unmanned aerial vehicle autonomous navigation, and relates to an intelligent unmanned aerial vehicle navigation method and system based on bimodal obstacle feature extraction. Aligning the information of the multi-modal sensor in time and space through a time-space alignment network; real-time pose information of the unmanned aerial vehicle is obtained through a Kalman filter fused with a positioning failure adaptive strategy; a double-flow obstacle feature extraction network is adopted to extract a composite environment vector containing dynamic obstacle features and static obstacle features from the multi-modal sensor data; training the SAC model based on the maximum entropy by adopting an improved reinforcement learning training method based on curriculum learning to obtain an SAC strategy network; and inputting the composite environment vector and the real-time pose information of the unmanned aerial vehicle into an SAC strategy network, and obtaining the optimal control of the unmanned aerial vehicle in an end-to-end manner, thereby having good path planning performance, and effectively improving the task execution capability and safety of the unmanned aerial vehicle in a complex environment.

Description

Technical Field

[0001] The present application belongs to the field of autonomous drone navigation technology, and relates to an intelligent drone navigation method and system based on dual-modal obstacle feature extraction. Background Art

[0002] With the advancement and development of science and technology, drones are becoming increasingly widely used in human production and daily life. As a fundamental technology in autonomous drone control, drone navigation has become a key focus in drone application research. To promote technological innovation in autonomous drone navigation and facilitate the application of related innovations in the drone field, the Drone Intelligent Perception Technology Competition has emerged. These competitions require drones to complete high-speed autonomous flight missions within complex racetracks according to system instructions.

[0003] Currently, most drone navigation methods utilize a planning-based approach to navigating three-dimensional scenes. This approach typically divides navigation tasks into three modules: mapping, motion planning, and low-level control. These modules work together to accomplish the drone's navigation and obstacle avoidance tasks. Mapping involves reconstructing environmental information using various sensors onboard the drone. Stereo cameras are widely used in the mapping module of drone navigation algorithms due to their small size and light weight. However, in complex environments such as low light and multiple obstacles, the perception capabilities of binocular cameras decline significantly. Mapping consumes excessive computational and memory resources, directly impacting the drone's autonomous flight. Motion planning involves front-end path search and back-end trajectory optimization based on an environmental map. Front-end path search algorithms can be categorized into three main categories. The first is sampling-based path search algorithms, such as the RRT algorithm and the RRT-star algorithm. These algorithms randomly sample the environment and generate a roadmap, then search for a feasible path from the starting point to the destination. However, as environmental complexity increases, sampling-based path search algorithms can suffer from increased computational complexity and the tendency to become trapped in local optimal solutions. Second is node-optimization-based path search algorithms, such as the A-star algorithm and the Dijkstra algorithm. These algorithms are the most widely used path search algorithms in mainstream UAV navigation algorithms and use heuristic information to find the optimal path. However, these algorithms suffer from computational inefficiency when there are many nodes and complex networks. Finally, there are artificial intelligence-based path search algorithms, such as swarm intelligence algorithms and reinforcement learning algorithms. These intelligent algorithms model path search as a constrained multi-objective optimization problem and typically use a single network to complete the UAV's path planning module. However, this algorithmic mechanism can suffer from sparse rewards in complex environments, making it difficult for the UAV to meet constraints. Numerous iterations are required to obtain the optimal path, significantly delaying training time. Back-end trajectory optimization transforms the path searched by the front-end into a smooth trajectory with continuous position, velocity, and acceleration through optimization algorithms such as B-spline optimization and MINCO, ensuring that it conforms to the basic dynamics of the drone and ensures safe flight. The current mainstream modular approach to drone navigation algorithms is highly interpretable and widely adopted. However, this approach suffers from the poor perception capabilities of a single sensor in complex environments and the high computational complexity of motion planning. This results in high lag in obstacle avoidance during high-speed flight, limiting the drone's autonomous flight. There is an urgent need to develop an intelligent drone path planning method and system that can cope with complex and changing environments and improve planning speed. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of the present application is to provide an intelligent drone navigation method based on dual-modal obstacle feature extraction, which solves the problems of poor single sensor perception ability, high computational complexity of modular navigation algorithm, and low security of no-positioning failure strategy in traditional drone autonomous navigation algorithms when facing complex and changing environments.

[0005] The embodiment of the present application also provides an intelligent drone navigation system based on dual-modal obstacle feature extraction.

[0006] The first embodiment of the present application provides an intelligent drone navigation method based on dual-modal obstacle feature extraction, the method comprising: Acquire multimodal sensor data, including point cloud data collected by a lidar, image data collected by a binocular camera, drone position information measured in real time by a GPS, and IMU data collected by an inertial measurement unit; Align multimodal sensor information in time and space through a spatiotemporal alignment network; The real-time position information of the UAV is obtained by integrating the Kalman filter with the positioning failure adaptive strategy; A dual-stream obstacle feature extraction network is used to extract a composite environment vector containing dynamic obstacle features and static obstacle features from multimodal sensor data; The maximum entropy-based SAC model is trained using a reinforcement learning training method based on curriculum learning to obtain a SAC policy network. The composite environment vector and the real-time pose information of the UAV are input into the SAC strategy network to obtain the optimal control of the UAV in an end-to-end manner.

[0007] Furthermore, the multimodal sensor data is aligned in time and space, including: Select the timestamp of the binocular camera as the unified time base, interpolate the point cloud data, and align it to the unified time base. Calculate the downsampling factor based on the sampling frequency of the IMU data and the binocular camera. Select a data value as the sampled data every downsampling factor data point, and align the IMU data to the unified time base. The feature map extracted from the time-aligned multimodal sensor data stream is input into a lightweight network to obtain the predicted spatial transformation parameters, and the feature map is transferred from the sensor coordinate system to the world coordinate system according to the spatial transformation parameters.

[0008] Furthermore, the Kalman filter integrated with the positioning failure adaptive strategy obtains the real-time posture information of the UAV, including: Determine whether the current GPS signal detected is invalid. If the GPS signal error does not exceed the set threshold, use the GPS signal as the positioning measurement value input of the Kalman filter; If the GPS signal error exceeds the set threshold, the GPS is in a failed state, triggering an emergency hover command and starting the laser SLAM positioning function; The weights of GPS and laser SLAM in the Kalman filter positioning measurement values are dynamically adjusted according to the failure degree of the GPS signal. The IMU data and positioning measurement values are input into the Kalman filter to obtain the real-time posture information of the UAV.

[0009] Furthermore, the dual-stream obstacle feature extraction network is used to extract a composite environment vector containing dynamic and static obstacle features from multimodal sensor data, including: Adopting a convolutional neural network-based target detection algorithm to identify dynamic obstacles in the environment and separate dynamic obstacles from static obstacles within the current range of the drone; Extract point cloud data of static obstacles, divide the environment space into 3D voxel grids with preset fixed memory, use hash tables to store space occupancy status, and construct a 3D voxel map reflecting the obstacle information around the drone; update the occupancy probability of each voxel based on the latest point cloud data; 3D ray projection is used to sample the 3D voxel map from the current position of the drone. Rays are projected in all directions in the horizontal and vertical planes with the drone as the center. The length vectors of all rays in the horizontal plane are recorded according to each projection angle in the vertical plane. Rays that exceed the maximum range are set to the length vector of the maximum range plus a small offset. The length vector is used as the characteristic of the static obstacle. Extract image data of dynamic obstacles, obtain depth images, and convert them into U-depth maps. From the U-depth maps, identify the outlines of dynamic obstacles and generate 3D bounding boxes to represent the height, width, and length of the dynamic obstacles. Input the image with 3D bounding box information into a long short-term memory network. Through temporal modeling, obtain the probability distribution of the future position of the dynamic obstacle as the dynamic obstacle feature. The features of static obstacles and dynamic obstacles are concatenated into a composite environment vector.

[0010] Furthermore, we adopt the reinforcement learning training method based on curriculum learning to train the SAC model based on maximum entropy and obtain the SAC policy network, including: The training of the SAC model is divided into two parts: the first part is to let the UAV focus on its navigation ability to fly in a straight line to the target point, and gradually introduce obstacle avoidance tasks; in the second part, the size of the environment and the number of obstacles are increased, and the complexity of the tasks is gradually improved to train the UAV's autonomous navigation and obstacle avoidance capabilities; by continuously training the SAC model until the network converges, the SAC policy network is obtained.

[0011] Furthermore, the composite environment vector and the real-time pose information of the UAV are input into the SAC strategy network to obtain the optimal control of the UAV in an end-to-end manner, including: The real-time posture information of the UAV and the composite environment vector are constructed into a state vector, which is then mapped into speed control instructions through the SAC strategy network. The UAV controls its own flight state through the speed control instructions to complete the end-to-end navigation task.

[0012] The second embodiment of the present application provides an intelligent drone navigation system based on dual-modal obstacle feature extraction, including: A data acquisition module is used to acquire multimodal sensor data, including point cloud data collected by a lidar, image data collected by a binocular camera, drone position information measured in real time by a GPS, and IMU data collected by an inertial measurement unit; The alignment module aligns the multimodal sensor information in time and space through a spatiotemporal alignment network; Positioning failure adaptive strategy module, used to start laser SLAM positioning when GPS fails, and use Kalman filter to fuse positioning information with IMU data to obtain real-time drone posture information; The obstacle feature extraction module is used to extract a composite environment vector containing dynamic obstacle features and static obstacle features from multimodal sensor data using a dual-stream obstacle feature extraction network; The SAC policy network training module uses a reinforcement learning training method based on curriculum learning to train a SAC model based on maximum entropy to obtain a SAC policy network; The generation module is used to input the composite environment vector and the real-time pose information of the UAV into the SAC strategy network to obtain the optimal control of the UAV in an end-to-end manner.

[0013] The technical solutions of the embodiments of this application have at least the following beneficial effects: In terms of data fusion, data-level multimodal fusion technology is used to integrate data from multiple sensors, including lidar, binocular cameras, inertial measurement units (IMUs), and global positioning systems (GPS). This fully leverages the complementary advantages of each sensor data, effectively overcoming the perception limitations of a single sensor in complex environments, improving the accuracy and robustness of environmental perception, and providing more accurate and reliable basic information for subsequent positioning, obstacle avoidance, and path planning. In terms of positioning strategy, the proposed positioning failure adaptive strategy is unique and innovative. Based on the Kalman filter, it can dynamically adjust the weights of GPS and laser simultaneous localization and mapping (SLAM) observation data in the positioning process according to the degree of GPS signal failure. This ensures that when the GPS signal is unstable or fails, the drone can still maintain a high level of positioning accuracy and reliability, thereby improving the positioning safety of the drone in complex environments. In obstacle avoidance processing, the constructed dual-stream obstacle feature extraction network achieves accurate classification and efficient feature extraction of both dynamic and static obstacles. Using distance vectors to characterize static obstacle features, the system discards traditional complex feature extraction methods, presenting static environmental information in a concise and intuitive manner, reducing computational complexity. A three-dimensional bounding box model combined with a long short-term memory network (LSTM) is used to capture the edges and motion trends of dynamic obstacles, enabling the system to predict the motion trajectory of dynamic obstacles during back-end path planning, thereby more proactively planning obstacle avoidance paths and significantly improving the success rate of obstacle avoidance. In terms of algorithm training, an improved reinforcement learning training method based on curriculum learning is introduced, enabling the SAC model to gradually improve its adaptability and generalization capabilities in complex and changing environments, achieving rapid algorithm convergence. While pursuing high returns, it also takes into account the diversity of flight strategies, ensuring that the drone can flexibly adjust its flight strategy according to different conditions in complex environments, and has good path planning performance, effectively improving the drone's mission execution capability and safety in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of an intelligent drone navigation method based on dual-modal obstacle feature extraction provided in an embodiment of the present application; Figure 2 This is a schematic diagram of path planning in a three-dimensional racing environment according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0016] like Figure 1The flowchart of the intelligent drone navigation method based on dual-modal obstacle feature extraction provided by the embodiment of the present application is shown. The intelligent drone navigation method based on dual-modal obstacle feature extraction includes the following steps: S101, acquiring multimodal sensor data, where the multimodal sensor data includes point cloud data collected by a lidar, image data collected by a binocular camera, drone position information measured in real time by a GPS, and IMU data collected by an inertial measurement unit; Initializing signal reception via the drone: The drone is equipped with a lidar, inertial measurement unit (IMU), binocular camera, and onboard computer. The onboard computer receives sensor data from the lidar, binocular camera, inertial measurement unit (IMU), and GPS, obtaining radar point clouds, binocular images, IMU data, and the drone's real-time location information. It also receives mission instructions from the starting point.

[0017] S102, aligning multimodal sensor information in time and space through a time-space alignment network; Spatiotemporal alignment of multimodal sensor data: The point cloud data collected by the lidar, the image data collected by the binocular camera, the drone position information measured in real time by the GPS, and the IMU data of the inertial measurement unit are aligned in time and space through the spatiotemporal alignment network to obtain complementary and spatiotemporally aligned sensor data.

[0018] S103, obtaining the real-time posture information of the UAV by integrating the Kalman filter with the positioning failure adaptive strategy; Positioning failure adaptive strategy: By real-time monitoring of the failure degree of GPS signals, the weights of GPS positioning information and laser SLAM positioning information in the positioning observation input of the Kalman filter are dynamically adjusted, and the position coordinates and three-axis attitude of the drone in the world coordinate system are obtained through the Kalman filter.

[0019] S104, extracting a composite environment vector including dynamic obstacle features and static obstacle features from the multimodal sensor data using a dual-stream obstacle feature extraction network; Dynamic obstacles in complex environments are detected using convolutional neural network target detection algorithms, such as the YOLO (You Only Look Once) algorithm. Obstacles are classified into two labels: static and dynamic. A dual-stream obstacle feature extraction network is used to extract the distance vector of static obstacles relative to the drone and the size and future position probability distribution of dynamic obstacles. A composite environment vector containing this information is then obtained through a fully connected layer.

[0020] S105 uses a reinforcement learning training method based on curriculum learning to train a maximum entropy-based SAC model to obtain a SAC policy network (Actor). The composite environment vector and the real-time pose information of the UAV are then input into the trained SAC policy network (Actor). This allows for optimal control of the UAV in an end-to-end manner, enabling rapid planning and obstacle avoidance in complex environments.

[0021] S106, inputting the composite environment vector and the real-time pose information of the UAV into the SAC strategy network to obtain the optimal control of the UAV in an end-to-end manner.

[0022] In one embodiment, aligning multimodal sensor information in time and space by using a time-space alignment network in S102 includes: The timestamp of the binocular camera is selected as the unified time base, and the point cloud data is interpolated and aligned to the unified time base. The downsampling factor is calculated based on the sampling frequency of the IMU data and the binocular camera. A data value is selected as the sampled data every downsampling factor data point, and the IMU data is aligned to the unified time base.

[0023] The feature map extracted from the time-aligned multimodal sensor data stream is input into a lightweight network to obtain the predicted spatial transformation parameters, and the feature map is transferred from the sensor coordinate system to the world coordinate system according to the spatial transformation parameters.

[0024] Specific implementation methods include: Select the timestamp of the binocular camera as the unified time base. Interpolate the point cloud data collected by the low-frequency lidar. Specifically, select the target time point , find the data values of adjacent time points in the lidar timestamp set, and calculate the median of the data values at both ends to obtain the target time point The data value of is used to align the point cloud data to a unified time base. The high-frequency IMU data is downsampled. Specifically, the downsampling factor is calculated based on the sampling frequency of the IMU data and the binocular camera. The downsampling factor calculation formula is as follows: , in is the sampling frequency of IMU, is the sampling frequency of the binocular camera, is the downsampling factor, every By selecting a data value from each data point as the sampled data, the IMU data can be aligned to a unified time base.

[0025] The feature map extracted from the time-aligned sensor data stream is input into the lightweight network R(2+1)D to obtain the predicted spatial transformation parameters. , spatial transformation parameters This includes translation and rotation. Based on the spatial transformation parameters, the warping function of the STAN (Spatial-Temporal Alignment Network) is used to resample the feature map. The feature map is transformed from the sensor coordinate system to the world coordinate system according to the spatial transformation parameters to obtain spatiotemporally aligned sensor data.

[0026] Among them, the lightweight network R(2+1)D is a spatiotemporal convolutional neural network, which decomposes the traditional 3D convolution into independent operations of 2D spatial convolution and 1D temporal convolution. The 2D spatial convolution extracts the spatial features of a single frame or adjacent frames, and the 1D temporal convolution captures the dynamic timing information in the frame sequence. This form of convolutional network can alleviate the difficulties of spatiotemporal joint modeling with lower parameter complexity, and more stably and quickly extract environmental features to obtain target values.

[0027] In one embodiment, obtaining the position and attitude information of the UAV by integrating the Kalman filter with the positioning failure adaptive strategy in S103 includes: The system checks whether the current GPS signal is faulty. If the GPS signal error does not exceed a set threshold, the GPS signal is used as the positioning measurement input for the Kalman filter. If the GPS signal error exceeds the set threshold, indicating that the GPS (here, the device) is faulty, an emergency hover command is triggered and the laser SLAM positioning function is activated. The system then enters program control and dynamically adjusts the weights of the GPS (here, the device) and laser SLAM in the Kalman filter positioning measurement based on the degree of GPS signal failure. The IMU data and positioning measurement are then input into the Kalman filter to obtain the drone's real-time position coordinates and attitude.

[0028] The Kalman filter is an efficient recursive optimization algorithm that dynamically estimates system states and reduces the effects of noise by fusing uncertain observational data with a system prediction model. Its core process consists of two phases: prediction and update. The prediction phase uses physical models (such as the equation of motion) to infer the current state and error covariance. The update phase, combined with sensor observations, modifies the predictions using a weighted average, with the weights determined by the confidence (covariance) of the predictions and observations. The final output is an optimal estimate, specifically inputting positioning measurements and IMU data to obtain the optimal pose estimate based on the drone's motion model.

[0029] Specifically, regarding the laser SLAM positioning mentioned above, the fast laser radar-inertial odometry package by tight-coupled iterated Kalman filter algorithm in laser SLAM is used to construct a local grid map, and the IMU data pre-integration is integrated to realize the position estimation of the UAV.

[0030] The specific strategies for GPS failure detection and dynamic adjustment of positioning measurement weights are as follows: The standard deviation of GPS signal positioning is calculated through a sliding window. The window size is 5 consecutive frames of data. When the standard deviation is continuously greater than 3 meters, the GPS is judged to be in a failure state. The GPS failure degree coefficient is calculated using the Sigmoid decay function. The calculation formula is as follows: , in is the continuous failure duration, is the failure judgment threshold, is the attenuation coefficient, according to the GPS failure degree coefficient Dynamically adjust the confidence weights of GPS positioning and laser SLAM positioning in the Kalman filter. The confidence weight distribution formula is as follows: , , in is the confidence weight of GPS, is the confidence weight of laser SLAM, Represents time. According to the confidence weights of GPS and laser SLAM, the positioning measurement value calculation formula can be used to obtain the positioning measurement input of the Kalman filter. The positioning measurement value formula is as follows: , in is the positioning measurement input, is the GPS positioning value, It is the positioning value of laser SLAM. After obtaining the positioning measurement, the real-time posture information of the drone can be obtained by fusing the IMU data through the Kalman filter algorithm. , the pose information includes the three-axis position of the drone in the world coordinate system , and the pitch of the drone in the body coordinate system , Roll ,yaw .

[0031] In one embodiment, in S104, a dual-stream obstacle feature extraction network is used to extract a composite environment vector containing dynamic and static obstacle features from multimodal sensor data, including: The convolutional neural network-based target detection algorithm YOLO is used to identify dynamic obstacles in the environment and separate dynamic obstacles from static obstacles within the current range of the drone. Extract point cloud data of static obstacles, divide the environment space into 3D voxel grids with preset fixed memory, use hash tables to store space occupancy status, and construct a 3D voxel map reflecting the obstacle information around the drone; update the occupancy probability of each voxel based on the latest point cloud data; 3D ray projection is used to sample the 3D voxel map from the current position of the UAV. Rays are projected in all directions in the horizontal and vertical planes with the UAV as the center. The length vectors of all rays in the horizontal plane are recorded according to each projection angle in the vertical plane. Rays that exceed the maximum range are set to the length vector of the maximum range plus a small offset. The length vector is used as the feature of the static obstacle. The length vector is the distance vector of the static obstacle relative to the UAV and represents the three-dimensional outline of the static obstacle.

[0032] Extract image data of dynamic obstacles, obtain depth images through stereo matching, and convert them into U-depth maps. From the U-depth maps, identify the outline of the dynamic obstacles and generate 3D bounding boxes to represent the height, width, and length of the dynamic obstacles. Input the image with 3D bounding box information into a long short-term memory network. Through time series modeling, obtain the probability distribution of the future position of the dynamic obstacles as the dynamic obstacle features. The features of static obstacles and dynamic obstacles are concatenated into a composite environment vector.

[0033] Specifically, the dual-stream obstacle feature extraction network extracts a composite environment vector containing dynamic and static obstacle features from multimodal sensor data, including: The drone's posture and multiple sensor fusion data are input into the target detection module, and the YOLO target detection algorithm based on convolutional neural network is used to identify dynamic obstacles in the environment and separate dynamic obstacles from static obstacles within the current range of the drone.

[0034] Extract the point cloud data of static obstacles and divide the environment space into 3D voxel grids with a preset fixed memory. The maximum number of voxels is determined by the environment scale and resolution. In this embodiment, 0.1 is used. The environment space is divided into voxels, and the space occupancy status is stored in a hash table to construct a 3D voxel map reflecting the obstacle information around the drone. Then, the occupancy probability of each voxel is recursively updated according to the Bayesian filter formula based on the latest point cloud data collected by the drone. The Bayesian update formula is as follows: , in Indicates that in all historical observation data (from time 1 to time t) under the condition voxel The posterior probability of being occupied, Indicates the current observation data Voxel The likelihood of being occupied, Indicates historical observation data The voxel below The occupancy probability of each voxel is recursively updated according to the above formula to obtain a real-time updated 3D voxel map.

[0035] After obtaining the 3D voxel map, the 3D ray projection technology is used to sample the 3D voxel map from the current position of the drone. The ray is projected in all directions in the horizontal and vertical planes with the drone as the center. Projection angle Record the length vectors of all rays in the horizontal plane The ray beyond the maximum range is set to the length of the maximum range plus a small offset. The maximum range here refers to the maximum boundary of the 3D voxel map. In the embodiment of this application, the maximum range is set to 10m and the offset is 1m. So far, the characteristics of static obstacles are obtained. express: , Indicates the total number of rays.

[0036] Extract image data containing dynamic obstacles, obtain depth images through stereo matching algorithms, and then use U-depth detectors to convert the original depth images into U depth maps. This U depth map can also be called a bird's-eye view depth map, which can intuitively reflect the height information in the scene and facilitate observation of the distribution and shape of obstacles from above. Then, a line grouping algorithm is used to identify the outline of the obstacles and generate 3D bounding boxes. , represents the height, width and length of the dynamic obstacle. The image with the bounding box information is input into the long short-term memory network (LSTM). The probability distribution of the future position of the dynamic obstacle can be obtained through time series modeling. So far, the characteristics of the dynamic obstacle are obtained. express: , , in Represents the dynamic obstacle characteristics, Representatives include Obstacle center position, represents the probability distribution of the future position of the obstacle, represents the 3D bounding box and The feature vector representing the current position of the drone, where the current position of the drone is From the drone's posture information Extract the first three digits from Indicates inclusion The center position of the obstacle, Indicates transposition. It should be noted that the current position of the drone represents the coordinates of the drone in the world coordinate system. ,The drone pose represents the three-axis coordinates and three-axis attitude of the drone in the world coordinate system . represents the roll angle, represents the yaw angle, Indicates the pitch angle.

[0037] After obtaining the characteristics of static obstacles and dynamic obstacles, their feature vectors are concatenated in terms of dimension to obtain a composite environment vector that reflects the complete environment state: , in Represents a composite environment vector. The composite environment vector includes the 3D outline of the static obstacle and the motion trajectory of the dynamic obstacle.

[0038] In one embodiment, in S105, a reinforcement learning training method based on curriculum learning is used to train a SAC model (Soft Actor-Critic) to obtain a SAC policy network (Actor), including: The course study divides the training of the SAC algorithm into two processes: Course Learning Process 1: Navigation and Obstacle Avoidance Learning in Phases. In the beginner stage, an obstacle-free environment is set up, such as a 40m x 40m obstacle-free environment, allowing the drone to learn basic navigation capabilities, namely how to directly navigate and reach a randomly set target point. In the intermediate stage, obstacles are randomly added to the previous obstacle-free environment, for example, around 10 obstacles are randomly added. This allows the drone to master basic navigation while learning simple obstacle avoidance strategies. In the advanced stage, obstacles are added while the environment remains unchanged, for example, around 20 obstacles are added. This allows the drone to learn efficient navigation and obstacle avoidance in more complex environments.

[0039] Course Learning Process 2: Simultaneous Expansion of Environment and Obstacles. The beginner stage sets up an environment of 20m*20m and adds, for example, 10 obstacles, to help the drone adapt to smaller environments and fewer obstacles. The intermediate stage expands the environment, for example, to 40m*40m and the number of obstacles to 20, allowing the drone to further enhance its adaptability to complex environments and obstacle avoidance capabilities. The advanced stage further expands the environment, for example, to 60m*60m, increasing the number of obstacles to 25, and setting dynamic obstacles, for example, 5, to simulate more complex environments and train the drone's advanced navigation and obstacle avoidance capabilities.

[0040] Through the above two training processes, the SAC model is gradually trained and updated until it converges to obtain the SAC policy network (Actor).

[0041] The specific SAC model update process includes: Define the state vector and action space , where the state vector Contains the current real-time position information of the drone 、Environmental conditions These two pieces of information are defined as: , Action Space It consists of the three-axis speed control of the drone, which is defined as: , in for Axis speed control, for Axis speed control, for Axis speed control, Represents the three-dimensional space where the drone is located.

[0042] Initialize the network parameters of the SAC model. The SAC model consists of a SAC policy network, a current value network, and a target value network. Specifically, the current value network and the target value network are dual-Q networks, each containing two independent Q networks. Here, Q represents value. This dual-Q network design can alleviate overestimation of Q values. Initialize the parameters of the SAC policy network (Actor) as , the current value network Critic1 network Critic2 network with current value network The parameters are ;Target value network Critic1 network and target value network Critic2 network The parameters of are copied from the current value network parameters.

[0043] Obtained from the experience replay pool after all preparations are initially completed State of the moment , input the SAC policy network to get an action and its logarithmic probability , input the state, action and logarithmic probability into the current value network Critic1 network Critic2 network with current value network Get two output values respectively, and take the smaller output value as the output value of the current value network .

[0044] Similarly, get the next state , input SAC strategy network to get the action of the next state and its logarithmic probability , input the state, action and logarithmic probability of the next moment into the target Critic1 network and target Critic2 network Get two output values and take the smaller output value as the output value of the target value network , the target is obtained by calculation value .

[0045] Then the output of the current value network With the goal value Calculate the mean square error and update the SAC policy network (Actor), current value network, and target value network according to the calculated mean square error until convergence.

[0046] The goal of the above operation value The calculation formula is: , in for value, is the future reward attenuation coefficient, For the The output of the target Critic network, for The state of the moment, is the output of the policy network as the target, is the current policy network, The entropy coefficient controls the regularization strength of entropy, balancing exploration and utilization. Here, the value is 0.2. It is an instant reward, which is calculated by the reward function. The reward calculation formula is as follows: , The first one is the speed bonus. is the target waypoint, is the current position of the drone. The closer the two distances are, the higher the reward. The second item is the static obstacle safety reward. For all projection angles in the vertical direction, For all projection distances in the horizontal plane for each projection angle, is the distance vector, where is the projection angle to the vertical plane, is the projection angle of the vertical plane The serial numbers of the corresponding multiple rays in the horizontal plane. The farther the drone is from the static obstacle, the higher the reward. The third item is the dynamic obstacle safety reward. is the number of dynamic obstacles, is the center position of the dynamic obstacle. The farther the distance, the higher the reward. The fourth item is the smooth reward item. is the speed at the current moment, is the speed at the previous moment. The smaller the speed difference, the greater the reward.

[0047] Through the above-mentioned iterative loop, the optimal SAC strategy is continuously found, and finally a trained SAC strategy network is obtained. Finally, the trained SAC strategy network can be used to obtain the speed control instructions of the drone to complete the navigation task.

[0048] like Figure 2 The figure shows the navigation diagram of the embodiment of the present application in a three-dimensional track scene. After the drone is initialized at the starting point, it receives the command of the starting point from the competition system and obtains sensor data from sensors such as lidar, binocular camera, inertial measurement unit, GPS, etc. After time-space alignment, positioning failure detection, and dual-stream obstacle feature extraction, the network can divide the environment 20 meters around the drone into static obstacles and dynamic obstacles and obtain their feature representations, as shown in the figure. Figure 2 The cuboid represents a static obstacle, and the cylinder represents a dynamic obstacle, which moves left and right at a constant speed within the track. The static obstacle cuboid is represented as a distance vector relative to the drone, and the dynamic obstacle is represented as a vector of a 3D bounding box, a center position, and a possible future distribution position. Then, the reinforcement learning path planning layer is entered, and the SAC optimal strategy trained by the SAC algorithm converts the waypoints into continuous speed control, such as Figure 2 The smooth dotted line in the figure shows that the drone has completed its mission.

[0049] On the other hand, an embodiment of the present application provides an intelligent drone navigation system based on dual-modal obstacle feature extraction, comprising: A data acquisition module is used to acquire multimodal sensor data, including point cloud data collected by a lidar, image data collected by a binocular camera, drone position information measured in real time by a GPS, and IMU data collected by an inertial measurement unit; The alignment module aligns the multimodal sensor information in time and space through a spatiotemporal alignment network; Positioning failure adaptive strategy module, used to start laser SLAM positioning when GPS fails, and use Kalman filter to fuse positioning information with IMU data to obtain real-time drone posture information; The obstacle feature extraction module is used to extract a composite environment vector containing dynamic obstacle features and static obstacle features from multimodal sensor data using a dual-stream obstacle feature extraction network; The SAC policy network training module uses a reinforcement learning training method based on curriculum learning to train a SAC model based on maximum entropy to obtain a SAC policy network; The generation module is used to input the composite environment vector and the real-time pose information of the UAV into the SAC strategy network to obtain the optimal control of the UAV in an end-to-end manner, realizing rapid planning and obstacle avoidance of the UAV in complex environments.

[0050] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0051] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. An intelligent UAV navigation method based on dual-modal obstacle feature extraction, characterized in that: The method comprises: Acquire multimodal sensor data, including point cloud data collected by a lidar, image data collected by a binocular camera, drone position information measured in real time by a GPS, and IMU data collected by an inertial measurement unit; Align multimodal sensor information in time and space through a spatiotemporal alignment network; The real-time position information of the UAV is obtained by integrating the Kalman filter with the positioning failure adaptive strategy; A dual-stream obstacle feature extraction network is used to extract a composite environment vector containing dynamic obstacle features and static obstacle features from multimodal sensor data; The maximum entropy-based SAC model is trained using a reinforcement learning training method based on curriculum learning to obtain a SAC policy network. The composite environment vector and the real-time pose information of the UAV are input into the SAC strategy network to obtain the optimal control of the UAV in an end-to-end manner.

2. The intelligent UAV navigation method based on dual-modal obstacle feature extraction according to claim 1 is characterized in that: The method of aligning multimodal sensor information in time and space by using a time-space alignment network includes: Select the timestamp of the binocular camera as the unified time base, interpolate the point cloud data, and align it to the unified time base. Calculate the downsampling factor based on the sampling frequency of the IMU data and the binocular camera. Select a data value as the sampled data every downsampling factor data point, and align the IMU data to the unified time base. The feature map extracted from the time-aligned multimodal sensor data stream is input into a lightweight network to obtain the predicted spatial transformation parameters, and the feature map is transferred from the sensor coordinate system to the world coordinate system according to the spatial transformation parameters.

3. The intelligent UAV navigation method based on dual-modal obstacle feature extraction according to claim 1 is characterized in that: The Kalman filter integrated with the positioning failure adaptive strategy obtains the real-time posture information of the UAV, including: Determine whether the current GPS signal detected is invalid. If the GPS signal error does not exceed the set threshold, use the GPS signal as the positioning measurement value input of the Kalman filter; If the GPS signal error exceeds the set threshold, the GPS is in a failed state, triggering an emergency hover command and starting the laser SLAM positioning function; The weights of GPS and laser SLAM in the Kalman filter positioning measurement values are dynamically adjusted according to the failure degree of the GPS signal. The IMU data and positioning measurement values are input into the Kalman filter to obtain the real-time posture information of the UAV.

4. The intelligent UAV navigation method based on dual-modal obstacle feature extraction according to claim 1 is characterized in that: The dual-stream obstacle feature extraction network is used to extract a composite environment vector containing dynamic and static obstacle features from multimodal sensor data, including: Adopting a convolutional neural network-based target detection algorithm to identify dynamic obstacles in the environment and separate dynamic obstacles from static obstacles within the current range of the drone; Extract point cloud data of static obstacles, divide the environment space into 3D voxel grids with preset fixed memory, use hash tables to store space occupancy status, and construct a 3D voxel map reflecting the obstacle information around the drone; update the occupancy probability of each voxel based on the latest point cloud data; 3D ray projection is used to sample the 3D voxel map from the current position of the drone. Rays are projected in all directions in the horizontal and vertical planes with the drone as the center. The length vectors of all rays in the horizontal plane are recorded according to each projection angle in the vertical plane. Rays that exceed the maximum range are set to the length vector of the maximum range plus a small offset. The length vector is used as the characteristic of the static obstacle. Extract image data of dynamic obstacles, obtain depth images, and convert them into U-depth maps. From the U-depth maps, identify the outlines of dynamic obstacles and generate 3D bounding boxes to represent the height, width, and length of the dynamic obstacles. Input the image with 3D bounding box information into a long short-term memory network. Through temporal modeling, obtain the probability distribution of the future position of the dynamic obstacle as the dynamic obstacle feature. The features of static obstacles and dynamic obstacles are concatenated into a composite environment vector.

5. The intelligent UAV navigation method based on dual-modal obstacle feature extraction according to claim 1 is characterized in that: The maximum entropy-based SAC model is trained using a reinforcement learning training method based on curriculum learning to obtain the SAC policy network, including: The training of the SAC model is divided into two parts: the first part is to let the UAV focus on its navigation ability to fly in a straight line to the target point, and gradually introduce obstacle avoidance tasks; in the second part, the size of the environment and the number of obstacles are increased, and the complexity of the tasks is gradually improved to train the UAV's autonomous navigation and obstacle avoidance capabilities; by continuously training the SAC model until the network converges, the SAC policy network is obtained.

6. The intelligent UAV navigation method based on dual-modal obstacle feature extraction according to claim 5 is characterized in that: By continuously training the SAC model until the network converges, the SAC policy network is obtained, including: Initialize the network parameters of the SAC model, the parameters of the SAC policy network, the parameters of the current value network Critic1 network and Critic2 network; the parameters of the target value network Critic1 network and the target value network Critic2 network; Obtained from the experience replay pool The state at the moment is input into the SAC policy network to obtain an action and logarithmic probability. The state, action and logarithmic probability are input into the current value network Critic1 network and the current value network Critic2 network to obtain two output values, and the smaller output value is taken as the output value of the current value network; Get the next state, input it into the SAC policy network to get the action and logarithmic probability of the next state, input the state, action and logarithmic probability of the next moment into the target Critic1 network and the target Critic2 network to obtain two output values, and take the smaller output value as the output value of the target value network; The target is obtained by calculating the output value of the target value network value; The output value of the current value network is compared with the target The mean square error is calculated and the SAC strategy network, current value network and target value network are updated according to the calculated mean square error until convergence.

7. The intelligent UAV navigation method based on dual-modal obstacle feature extraction according to claim 5 is characterized in that: The composite environment vector and the real-time pose information of the UAV are input into the SAC strategy network to obtain the optimal control of the UAV in an end-to-end manner, including: The real-time posture information of the UAV and the composite environment vector are constructed into a state vector, which is then mapped into speed control instructions through the SAC strategy network. The UAV controls its own flight state through the speed control instructions to complete the end-to-end navigation task.

8. An intelligent UAV navigation system based on dual-modal obstacle feature extraction, characterized in that: include: A data acquisition module is used to acquire multimodal sensor data, including point cloud data collected by a lidar, image data collected by a binocular camera, drone position information measured in real time by GPS, and IMU data collected by an inertial measurement unit; The alignment module aligns the multimodal sensor information in time and space through a spatiotemporal alignment network; Positioning failure adaptive strategy module, used to start laser SLAM positioning when GPS fails, and use Kalman filter to fuse positioning information with IMU data to obtain real-time drone posture information; The obstacle feature extraction module is used to extract a composite environment vector containing dynamic obstacle features and static obstacle features from multimodal sensor data using a dual-stream obstacle feature extraction network; The SAC policy network training module uses a reinforcement learning training method based on curriculum learning to train a SAC model based on maximum entropy to obtain a SAC policy network; The generation module is used to input the composite environment vector and the real-time pose information of the UAV into the SAC strategy network to obtain the optimal control of the UAV in an end-to-end manner.

Images