Unmanned aerial vehicle autonomous path navigation system based on multi-modal fusion and adaptive optimization and algorithm thereof

Abstract

The invention relates to an unmanned aerial vehicle autonomous path navigation system based on multi-modal fusion and adaptive optimization and an algorithm thereof. The system comprises a multi-modal sensor module, a data fusion processing module, an adaptive algorithm processing module and a flight control module. The method comprises the following steps: step 1, data acquisition; step 2, data fusion; step 3, algorithm processing; and 4, controlling the flight of the unmanned aerial vehicle. The method has the advantages that multiple sensors such as a visual camera, a laser radar and a millimeter wave radar are integrated through deep fusion of multi-modal sensors, a fusion algorithm based on deep learning and a Transform architecture is adopted, weights are automatically distributed through an attention mechanism, deep data fusion is achieved, and the accuracy and accuracy of data fusion are improved. Compared with a single sensor or a simple data fusion mode, the problem that the single sensor is easily interfered by the environment is effectively solved, and the obstacle recognition accuracy and the environment modeling precision are greatly improved under the complex environments of low illumination, strong smoke, electromagnetic interference and the like.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) navigation and control technology, specifically to a multimodal sensor data fusion and adaptive optimization algorithm. Technical Background

[0002] Currently, UAV autonomous navigation technology has made some progress, but many problems still exist: In environmental modeling, most existing technologies rely on single sensors that are susceptible to environmental interference; for example, visual sensors cannot accurately acquire information in low light or heavy smoke environments. Regarding path planning, traditional algorithms struggle to quickly generate optimal paths that balance efficiency and safety when facing complex terrain or dynamic obstacles. During obstacle avoidance, the recognition and response speed for small, concealed obstacles is insufficient. Therefore, there is an urgent need for an autonomous navigation technology for UAVs that can achieve high-precision environmental perception, efficient path planning, and rapid obstacle avoidance in complex and changing environments.

[0003] In published patent applications, for example, Chinese Invention Patent Application Publication No. CN119268700A discloses a method and system for UAV navigation and positioning based on big data. The method includes: acquiring UAV location data, sensor data, and environmental data; constructing a real-time map and performing real-time positioning using a SLAM algorithm based on the location and sensor data; dividing the real-time map into sub-regions and obtaining a risk score using cluster analysis; obtaining the path time between sub-regions based on the environmental data and risk score; establishing a fitness function based on the risk score and path time; establishing a path planning model based on the fitness function; obtaining offset data based on the sensor data; establishing a path compensation strategy and performing path correction based on the offset data; learning the path compensation strategy and path planning model to obtain a UAV navigation model; and optimizing the UAV navigation model using a sine / cosine optimization algorithm. This invention is applicable to various UAV application scenarios and promotes the widespread application of UAV technology in complex environments.

[0004] For example, Chinese invention patent application CN201910661936.1 discloses an image recognition-based unmanned aerial vehicle (UAV) navigation system and method. The UAV navigation system includes a backend control module, a location acquisition module, a command receiving module, and a flight trajectory verification module. The backend control module sends commands to the UAV. The command receiving module receives the commands sent by the backend control module and controls the location acquisition module and the flight trajectory verification module to operate according to the commands. The location acquisition module acquires the UAV's location and feeds it back to the backend control module. The flight trajectory verification module verifies the correctness of the UAV's flight trajectory and feeds the verification result back to the backend control module. The location acquisition module includes a location image acquisition module, a location feature image processing module, an altitude determination module, a location image similarity comparison module, and a location output module.

[0005] For example, Chinese invention patent application publication number CN107289948A discloses a UAV navigation system based on multi-sensor fusion, which achieves more accurate UAV navigation compared to traditional technologies, improving the level of intelligence and navigation accuracy. The system includes: a binocular module, a real-time positioning and mapping module, an ultrasonic module, an inertial measurement module, a barometer module, a storage module, a human-computer interaction module, a navigation module, an extended Kalman filter module, and a GPS module; the binocular module is signal-connected to the real-time positioning and mapping module; the real-time positioning and mapping module, ultrasonic module, inertial measurement module, and barometer module are signal-connected to the extended Kalman filter module; the extended Kalman filter module is signal-connected to the GPS module and the storage module; and the storage module is signal-connected to the navigation module and the human-computer interaction module. Furthermore, this invention also discloses a corresponding UAV navigation method based on multi-sensor fusion, suitable for precise UAV navigation.

[0006] None of the aforementioned invention patent applications have been able to solve the problem that the systems or traditional algorithms disclosed in the path planning field are unable to quickly generate the optimal path that balances efficiency and safety when faced with complex terrain or dynamic obstacles; and that the recognition and response speed for small-sized, hidden obstacles during obstacle avoidance is insufficient.

[0007] Based on the defects or problems of the existing technology, there is an urgent need to provide an autonomous path navigation system and algorithm for unmanned aerial vehicles (UAVs) based on multimodal fusion and adaptive optimization. Summary of the Invention

[0008] This invention proposes an autonomous path navigation system for unmanned aerial vehicles (UAVs) based on multimodal fusion and adaptive optimization, and its algorithm.

[0009] The UAV autonomous path navigation system based on multimodal fusion and adaptive optimization includes:

[0010] The system comprises a multimodal sensor module, a data fusion processing module, an adaptive algorithm processing module, and a flight control module. The multimodal sensor module is sequentially and electrically connected to the data fusion processing module, the adaptive algorithm processing module, and the flight control module. The multimodal sensor module integrates a visual camera, LiDAR, millimeter-wave radar, ultrasonic sensor, and inertial measurement unit. The visual camera employs a high dynamic range imaging technology of 100-140dB, enabling it to clearly capture images under different lighting conditions, providing high-quality data for subsequent image feature extraction. The LiDAR uses a solid-state hybrid LiDAR to acquire 3D point cloud data of the surrounding environment, depicting the position and outline of obstacles. The millimeter-wave radar has high-resolution detection capabilities, enabling real-time monitoring of obstacle speed and orientation information, and is particularly suitable for detecting dynamic obstacles. The ultrasonic sensor is used for detecting nearby obstacles, assisting the visual camera, LiDAR, and millimeter-wave radar in achieving comprehensive perception of the UAV's surrounding environment. The inertial measurement unit measures the UAV's linear acceleration, angular velocity, and other motion information in real time, providing basic data for attitude calculation and position estimation.

[0011] The data fusion processing module uses a deep learning-based multimodal data fusion algorithm to preprocess and fuse the data collected by each sensor. The data fusion processing module receives the raw data from the multimodal sensor module, performs preprocessing operations such as filtering and noise reduction on the data to remove noise interference, and inputs the preprocessed data into the fusion network based on the Transformer architecture. The attention mechanism automatically allocates the weights of each modality of data to achieve deep data fusion.

[0012] The adaptive algorithm processing module includes adaptive optimization algorithms such as environment modeling, path planning, and obstacle avoidance. The adaptive algorithm processing module receives fused data from the data fusion processing module, performs environment modeling, path planning, and obstacle avoidance processing respectively, and dynamically adjusts the algorithm parameters according to the environment model and flight mission requirements to adapt to different environments and mission needs.

[0013] The flight control module, based on the output of the adaptive algorithm processing module, precisely controls the drone's flight attitude and speed.

[0014] The flight control module receives flight commands generated by path planning and obstacle avoidance algorithms, and achieves stable flight and autonomous navigation of the UAV by controlling the motor speed, servo angle and other actuators.

[0015] Furthermore, the multimodal sensor module, according to the sampling frequency: visual camera: 30-60 FPS, lidar: 10-20 Hz, millimeter-wave radar: 20-50 Hz, ultrasonic sensor: 50-100 Hz, collects images, point clouds, speed, and orientation data of the environment around the UAV, as well as the motion information of the UAV itself, and transmits these raw data to the data fusion processing module.

[0016] Secondly, this application provides an optimization algorithm for an autonomous path navigation system for unmanned aerial vehicles (UAVs) based on multimodal fusion and adaptive optimization, specifically including:

[0017] Step 1, data acquisition: Using a multimodal sensor module, images, point clouds, speed, azimuth, and other data of the environment around the UAV are collected simultaneously at a certain sampling frequency, as well as the motion information of the UAV itself. These raw data are then transmitted to the data fusion processing module.

[0018] Step 2, data fusion: The data fusion processing module preprocesses the collected raw data to remove noise and outliers, and then inputs the preprocessed data into a fusion network based on the Transformer architecture. The weights of each modality are calculated through the attention mechanism, and the data of different modalities are deeply fused to obtain fused data containing rich environmental information, and then transmitted to the adaptive algorithm processing module.

[0019] Step 3, Algorithm Processing: The adaptive algorithm processing module performs multimodal environment modeling, path planning, and obstacle avoidance based on the fused data. The multimodal environment modeling algorithm: constructs a 3D environment model using the fused data, updates the state of each voxel in the model in real time, and distinguishes between passable and impassable areas; based on the target location information and the environment model, it uses an adaptive path planning algorithm to generate the optimal flight path; the obstacle avoidance algorithm monitors obstacles in the environment in real time, and when an obstacle is detected, it adopts an appropriate obstacle avoidance strategy based on the obstacle's threat level, generating an obstacle avoidance path. Specifically: multimodal refers to the fusion mode of various types of sensor data, including: visual modality: image / video data from a visual camera containing high dynamic range imaging information; point cloud modality: 3D environmental point cloud data from LiDAR, used to construct spatial obstacle outlines; radar modality: target velocity and azimuth information from millimeter-wave radar, used for dynamic obstacle detection; sonar modality: near-range ranging data from ultrasonic sensors, assisting in near-field obstacle perception; and inertial modality: attitude information from the UAV itself, including motion state data such as acceleration and angular velocity from the IMU.

[0020] Step 4: UAV flight control. The flight control module receives flight commands generated by path planning and obstacle avoidance algorithms. By controlling the UAV's actuators, it adjusts the UAV's flight attitude and speed, enabling the UAV to fly along the planned path. At the same time, it continuously provides feedback on the UAV's actual flight status, forming a closed-loop control system to ensure the UAV's flight stability and safety.

[0021] Furthermore, in step 3, the multimodal fusion environment modeling algorithm employs deep learning-based multimodal data fusion technology for environment modeling:

[0022] First, visual feature extraction is performed on sensor data from different modalities to obtain information such as the shape, texture, distance, and speed of obstacles; visual image feature extraction:

[0023] Convolutional Neural Networks (CNNs) are used to extract features from visual images to obtain information such as the shape and texture of obstacles. Let the input visual image be I∈R. H*W*C Where H, W, and C represent the height, width, and number of channels of the image, respectively, the feature representation of the visual image is obtained through a series of convolutional layers, pooling layers, and activation functions: F v ∈R h*w*c , where h, w, and c are the height, width, and number of channels of the feature map, respectively;

[0024] Then, these feature information are input into a fusion network based on the Transformer architecture, and the weights of each modality data are automatically assigned through the attention mechanism to achieve deep data fusion;

[0025] Finally, the voxel octree algorithm is used to construct a three-dimensional environment model. The state of each voxel in the model is updated in real time according to the fused data to distinguish between passable and impassable areas.

[0026] LiDAR point cloud feature extraction: LiDAR data is processed using a point cloud processing algorithm to obtain the precise distance and 3D contour of obstacles. Let the input LiDAR point cloud data be P = {p1, p2, ..., p...} n}, where p i =(x i ,y i ,z i Let F be the 3D coordinates of the i-th point. A deep learning-based point cloud feature extraction network, such as PointNet or PointNet++, is used to convert the point cloud data into a feature vector F. l ∈R d , where d is the dimension of the feature vector;

[0027] Millimeter-wave radar data processing: Millimeter-wave radar data is converted into the speed and orientation information of obstacles through signal processing algorithms. Let the input millimeter-wave radar data be R = {r1, r2, ..., r...} m}, where r j =(v j ,θ j Let f(j) represent the velocity and orientation information of the j-th obstacle. The millimeter-wave radar data is then converted into a feature vector F using the feature mapping function fR. m ∈R k , where k is the dimension of the feature vector;

[0028] Ultrasonic sensor data processing: The ultrasonic sensor measures proximity information of surrounding objects in real time. Let the measurement data be U = {u1, u2, ..., u...} q}, u i Let represent the distance value of the i-th measurement. First, process it using the Kalman filter algorithm. The prediction equation of the Kalman filter is as follows (1):

[0029]

[0030] P k|k-1 =AP k-1|k-1 A T +Q......(1)

[0031] The updated equation is as follows (2):

[0032] K k =P k|k-1 H T HP k|k-1 H T +R) -1

[0033]

[0034] P k|k =(IK k H)P k|k-1 ......(2),

[0035] in, and These are the predicted state and the updated state, respectively; P k∣k-1 and P k∣k A is the corresponding covariance matrix; B is the state transition matrix; C is the control matrix. H is the control variable; H is the observation matrix; Q is the process noise covariance matrix; R is the observation noise covariance matrix; K k Kalman gain;

[0036] After filtering, a stable near-range feature vector is obtained: Fu ∈R l l is the dimension of the feature vector;

[0037] Inertial Measurement Unit (IMU) Data Processing: The accelerometer and gyroscope data acquired by the IMU are respectively A acc ={a x ,a y ,a z} and A gyro ={ω x ,ω y ,ω z The complementary filtering algorithm is used to fuse the two data to eliminate drift error. The complementary filtering is as follows (3):

[0038]

[0039] θ z =θ z +ω z Δt......(3),

[0040] Where θx, θy, θz are the fused attitude angles, α is the complementary filter coefficient, and Δt is the sampling time interval. After processing, the accurate attitude and motion feature vectors of the UAV are obtained: F imu ∈R m , where m is the dimension of the feature vector.

[0041] Furthermore, in step 2, the multimodal data fusion involves fusing the extracted visual image features F v LiDAR point cloud features F l Millimeter-wave radar characteristics F m Ultrasonic sensor characteristics F u and IMU features F imu The input is fed into a Transformer-based fusion network, which consists of multiple encoder layers. Each encoder layer includes a multi-head self-attention mechanism and a feedforward neural network. Through the multi-head self-attention mechanism, the network can automatically learn the correlations and weights between the data from different modalities. Let the fused feature vector be F. f ∈R D Where D is the dimension of the fused feature vector, and the fusion process is represented as: F f =Transformer, where the parameter of Transformer is F v ,F l ,F m ,F u ,F imu .

[0042] Further, in step 3, the adaptive environment model is modeled as follows: a three-dimensional environment model is constructed using a voxel octree algorithm, dividing the space around the UAV into multiple voxels. Each voxel has a state value indicating whether there are obstacles within that voxel. Based on the fused feature vector F... f The state of each voxel is determined by a classifier. Let the state of the voxel be s∈{0,1}, where 0 represents a passable area and 1 represents an impassable area. At the same time, combined with the UAV attitude and position information provided by the IMU, the position and attitude of the UAV itself in the model are accurately determined. The classification process can be represented as: s=classifier, and the position and attitude of the UAV in the model are updated according to the IMU data.

[0043] Furthermore, in step 3, the adaptive path planning algorithm, based on the traditional A* algorithm, introduces adaptive weight adjustment and scenario optimization strategies. Through online learning algorithms, it dynamically adjusts the weight coefficients in the cost function according to the environmental complexity and the flight status of the UAV, so that the algorithm can balance safety and efficiency. At the same time, it designs special heuristic functions for different scenarios to guide the algorithm to plan a better path.

[0044] The traditional dynamic cost function of the A* algorithm is F(n) = G(n) + H(n), where G(n) is the actual cost from the starting point to node n, calculated based on physical distance, flight time, or energy consumption; H(n) is the estimated cost from node n to the target point, using a scene adaptive function; and C(n) is the environmental complexity cost, calculated based on obstacle density, dynamic obstacle speed, etc. In this algorithm, an adaptive weight adjustment strategy is introduced, and the cost function becomes F(n) = αG(n) + βH(n) + γC(n), where α, β, and γ are weight coefficients, and α + β + γ = 1.

[0045] Dynamic weight adjustment model: This model employs a reinforcement learning framework for weight optimization. By constructing a state space, action space, and reward function, and utilizing the Q-learning algorithm, it dynamically adjusts the weight coefficients, enabling the UAV to adaptively optimize its path planning strategy under different environments and mission conditions.

[0046] The detailed definition and calculation of the state space S: The state space S describes the environment in which the UAV is located and its own state information. It consists of two main parts: environmental complexity indicators and UAV state indicators.

[0047] Environmental complexity metric: Obstacle density. The space surrounding the drone is divided into multiple identical cubic cells. The volume ratio occupied by obstacles in each cell is calculated, and then the average ratio across all cells is taken to obtain the obstacle density. Let there be N cells in total, and the obstacle volume in the i-th cell be V. iThe total volume of the cells is V. total Then the obstacle density D obs The calculation is as follows (4):

[0048]

[0049] Equation (4) above shows that the higher the obstacle density, the more complex the environment, and the higher the safety requirements for path planning;

[0050] Distribution entropy measures the uniformity of obstacle distribution in space. Based on the concept of Shannon entropy, let p be the probability of an obstacle appearing in different regions. j Let j represent different regions, then the distribution entropy is as follows (5):

[0051]

[0052] Equation (5) above shows that the greater the distribution entropy, the more dispersed and irregular the distribution of obstacles, and the more difficult the path planning becomes.

[0053] The drone status indicators include:

[0054] Drone speed, the current flight speed of a drone, includes three components, namely v x v y v z Combine them into a velocity vector

[0055] Drone battery level: E, representing the remaining percentage of the drone's battery charge. rem It indicates that when the battery is low, route planning should prioritize the shortest path to complete the task or return to the charging point as quickly as possible.

[0056] Remaining task time: Calculated based on the total task time requirement and the time already spent executing. Let the total task time be T. total The execution time is T. exec The remaining time for the task is: T rem =T total -T exec When time is running out for a task, the algorithm needs to choose a more efficient path while ensuring safety.

[0057] Integrating all the above indicators together forms the state vector: S = [D obs E dis ,v x ,v y ,v z E rem ,T rem This comprehensively reflects the current environment and status of the drone;

[0058] The specific content and constraints of action space A: Action space A is defined as a weight coefficient vector [α, β, γ]. These weight coefficients are used to adjust the proportion of each part in the path planning cost function: F(n) = α·G(n) + β·H(n) + γ·C(n), where:

[0059] α controls the actual cost G from the starting point to node n. The weight (n) of G(n) is usually calculated based on physical distance, flight time or energy consumption. When α is large, the algorithm will be more inclined to choose the path with lower actual cost and focus on the economic efficiency of the path.

[0060] β controls the weight of the heuristic estimation cost H(n) from node n to the target. H(n) adopts a scene adaptive function. Increasing the value of β can speed up the search speed of the algorithm.

[0061] γ controls the weight of the environmental complexity cost C(n), which is calculated based on obstacle density and dynamic obstacle speed. Increasing γ enables the algorithm to focus more on avoiding dangerous areas in complex environments, thus ensuring flight safety.

[0062] To ensure the rationality of the cost function, the weight coefficients satisfy α+β+γ=1, and α≥0, β≥0, γ≥0. The UAV adjusts the values ​​of α, β, and γ to change the focus of path planning to adapt to different environments and mission requirements.

[0063] Multi-dimensional calculation method of reward function R: The reward function R is used to evaluate the state change effect of the UAV after performing the action. It comprehensively considers multiple dimensions such as path length, flight time, energy consumption, and safety margin. Its calculation formula is as follows (6):

[0064] R=w1·ΔL+w2·ΔT+w3·ΔE+w4·ΔS

[0065] Where: w1, w2, w3, and w4 are the weight coefficients of each dimension indicator, set according to the priority and importance of the task, and w1 + w2 + w3 + w4 = 1; ΔL represents the change in path length, which is the current path length L. cur Compared to the previous path length L pre The difference, i.e., ΔL = L cur -L pre Shortening the path length can improve task execution efficiency. When ΔL < 0, a positive reward will be given; otherwise, a negative reward will be given.

[0066] ΔT represents the change in flight time, and is calculated similarly to path length: ΔT = T cur -T pre Reducing flight time allows for faster mission completion; similarly, a positive reward is obtained when ΔT < 0.

[0067] ΔE represents the change in energy consumption, ΔE = E cur -E pre Reducing energy consumption can extend the drone's endurance, and a positive reward is given when ΔE < 0.

[0068] ΔS represents the change in safety margin, which is assessed by calculating factors such as the distance between the drone and obstacles and hazardous areas. Let S be the safety margin index. index Then ΔS = S index (cur)-S index (pre). When the safety margin increases, ΔS>0, and a positive reward is given to encourage the drone to choose a safer path;

[0069] The iterative update process of the Q-learning algorithm;

[0070] The Q-learning algorithm iteratively updates the optimal strategy through continuous trial and error and learning, and its expression is as follows (6):

[0071]

[0072] Where: Q(s,a) represents the Q value of performing action a in state s, which represents the expected cumulative reward of the action in the current state; η is the learning rate, which ranges from [0,1] and is used to control the degree of adoption of new information in each update. The closer η is to 1, the more the algorithm focuses on the current reward, and the faster the learning speed is but may not be stable; the closer η is to 0, the more the algorithm relies on previous experience, the smoother the learning process is but the slower the convergence speed is.

[0073] R(s,a) is the immediate reward obtained after performing action a in state s, which is calculated by the reward function;

[0074] γ is a discount factor, which ranges from [0,1] and is used to measure the importance of future rewards. The closer γ is to 1, the more the algorithm values ​​future rewards and considers long-term benefits; the closer γ is to 0, the more it focuses on immediate rewards.

[0075] s′ is the new state transitioned to after performing action a, and a′max Q(s′,a′) represents the maximum value of Q among all possible actions in the new state s′, which represents the optimal expected cumulative reward that can be obtained from the new state;

[0076] Furthermore, in step 3, the specific iterative update procedure is as follows:

[0077] In the initial state s0, the drone selects action a0 according to the ∈-greedy policy. The ∈-greedy policy randomly selects actions with a probability of ∈ and selects the action with the largest current Q value with a probability of 1-∈. By adjusting the value of ∈, the relationship between exploring new actions and utilizing existing experience can be balanced.

[0078] After performing action a0, the drone transitions to a new state s1 and receives an immediate reward R(s0,a0). The value of Q(s0,a0) is then updated according to the Q-learning formula.

[0079] Repeat the above steps, continuously select actions, execute actions, obtain rewards and update Q value in different states. As the number of iterations increases, Q value gradually converges, and the drone can learn the optimal action in various states, that is, the optimal path planning strategy.

[0080] Based on the above description of the state space, action space, reward function, and Q-learning algorithm, the dynamic adjustment model of weight coefficients enables UAVs to autonomously learn and optimize the weight coefficients of path planning under complex environments and changing mission requirements, thereby achieving efficient and safe path navigation.

[0081] Furthermore, the scenario optimization strategy involves designing dedicated heuristic functions for different scenarios:

[0082] Urban canyon environment: as shown in equation (7):

[0083] H city (n)=w1·d euclid (n,g)+w2·h building (n)+w3·t shadow (n)......(7),

[0084] Where: d euclid (n, g): Euclidean distance from node n to the target, h building (n): Average height of buildings surrounding node n, t shadow (n): Probability that node n is located in the shadow area of ​​a building; w1, w2, w3: Environmental adaptive weights, dynamically adjusted by an SVM classifier;

[0085] Open terrain environment: as shown in formula (8):

[0086] H open (n)=w1·d euclid (n, g) + w2·v wind (n).d euclid (n,g)......(8),

[0087] Where: v wind(n)This represents the estimated wind speed at node n, obtained through the fusion of meteorological data and sensor data.

[0088] Complex indoor environments: as shown in equation (9):

[0089]

[0090] Where: d obs (n,i): The distance from node n to the i-th obstacle, and k represents the number of obstacles within the field of view.

[0091] Furthermore, in step 3, the obstacle avoidance algorithm is as follows:

[0092] Rapid obstacle avoidance is achieved based on an improved Dijkstra algorithm and a dynamic risk assessment mechanism. When the multimodal sensor detects an obstacle, the threat level of the obstacle to the UAV is first calculated using a dynamic risk assessment algorithm. For high-threat obstacles, a bidirectional search Dijkstra algorithm is used to quickly generate an obstacle avoidance path. For low-threat obstacles, a local path adjustment strategy is adopted to fine-tune the original path, reducing the computational load of path planning. At the same time, reinforcement learning algorithms are used to optimize the obstacle avoidance strategy, enabling the UAV to autonomously select the optimal obstacle avoidance method in different scenarios. The specific steps and calculation formulas are as follows:

[0093] Dynamic risk assessment: Taking into account factors such as the obstacle's speed, distance, and size, as well as the drone's flight speed and direction, the threat level R of the obstacle to the drone is calculated. Let the obstacle's speed be v. o The distance is d o Size is s o The drone's flight speed is v u The direction is θ u The formula for calculating the threat level is as follows (10):

[0094]

[0095] Where w1, w2, w3, and w4 are weighting coefficients, and θ O The direction of the obstacle's movement;

[0096] Obstacle avoidance strategy selection: For high-threat level obstacles, when R>R th (R th When the threat level threshold is reached, the Dijkstra algorithm with bidirectional search is adopted. The search starts and ends at the current position S of the UAV and the nearest unreachable node T in the original path plan. The search starts from both S and T. When the two search paths meet, the obstacle avoidance path is obtained.

[0097] Low threat level obstacles: when R≤Rth In this case, a local path adjustment strategy is adopted. Based on the original path, the path is finely adjusted according to the position and direction of movement of the obstacles, so that the drone can avoid the obstacles;

[0098] Reinforcement Learning Optimization: The obstacle avoidance strategy is optimized using reinforcement learning algorithms. The obstacle avoidance process of the UAV is regarded as a Markov Decision Process (MDP), defining the state space S, action space A, and reward function R. Through continuous simulation training, the UAV learns the optimal obstacle avoidance strategy. Let the state be s∈S and the action be a∈A, then the goal of reinforcement learning is to maximize the cumulative reward as shown in equation (11):

[0099]

[0100] Where π is the strategy, γ is the discount factor, and T is the number of time steps.

[0101] Compared with existing technologies in this field, the UAV autonomous path navigation system and algorithm based on multimodal fusion and adaptive optimization described in this invention have the following superior technical effects:

[0102] 1. The UAV autonomous path navigation system based on multimodal fusion and adaptive optimization described in this invention integrates multiple sensors such as visual cameras, lidar, and millimeter-wave radar through deep fusion of multimodal sensors. It adopts a fusion algorithm based on deep learning and Transformer architecture and automatically allocates weights through an attention mechanism to achieve deep data fusion. Compared with single sensor or simple data fusion methods, it effectively overcomes the problem of single sensor being susceptible to environmental interference. In complex environments such as low light, strong smoke, and electromagnetic interference, it significantly improves the accuracy of obstacle recognition and environmental modeling.

[0103] 2. The UAV autonomous path navigation algorithm based on multimodal fusion and adaptive optimization described in this invention employs an adaptive environment modeling algorithm, utilizes a voxel octree algorithm to construct a 3D environment model, and updates the model in real time based on the fused multimodal data. Compared with traditional environment modeling methods, it can more accurately and dynamically reflect environmental changes, providing more reliable basic data for subsequent path planning and obstacle avoidance.

[0104] 3. The UAV autonomous path navigation algorithm based on multimodal fusion and adaptive optimization described in this invention, through dynamic adaptive path planning, introduces an online learning algorithm to dynamically adjust the weight coefficients in the cost function on the basis of the traditional A* algorithm, and automatically balances the safety and efficiency of path planning by comprehensively considering environmental complexity and UAV flight status. At the same time, it designs special heuristic functions for different scenarios such as urban canyons and open terrain, enabling UAVs to plan better paths in various environments. Compared with traditional path planning algorithms, it significantly improves path search speed, shortens path length, and reduces flight energy consumption.

[0105] 4. The UAV autonomous path navigation algorithm based on multimodal fusion and adaptive optimization described in this invention employs a hierarchical rapid obstacle avoidance strategy. Based on an improved Dijkstra algorithm and combined with a dynamic risk assessment mechanism, it implements different obstacle avoidance strategies according to the obstacle threat level. For high-threat obstacles, it uses bidirectional search to quickly generate obstacle avoidance paths, while for low-threat obstacles, it performs local path fine-tuning. Simultaneously, it utilizes reinforcement learning algorithms to continuously optimize the obstacle avoidance strategy. This innovation significantly shortens the UAV's response time to obstacles and significantly improves the success rate of detecting and avoiding small obstacles, greatly enhancing flight safety.

[0106] 5. The UAV autonomous path navigation system and its algorithm based on multimodal fusion and adaptive optimization described in this invention comprehensively considers multiple factors in its scenario-based and intelligent technology application solution. It achieves full-process adaptation and intelligence from environmental perception and path planning to obstacle avoidance, and can be widely applied in multiple fields such as forest fire monitoring, urban logistics distribution, agricultural plant protection, and power line inspection. It meets the autonomous navigation needs of UAVs in different scenarios and expands the application boundaries and practicality of UAVs.

[0107] 6. The UAV autonomous path navigation system and algorithm based on multimodal fusion and adaptive optimization described in this invention, through the combination of high-precision environmental perception multimodal sensor fusion and deep learning algorithms, improves the UAV's obstacle recognition accuracy to over 98% and environmental modeling accuracy by 30% in complex environments such as low light, strong smoke, and electromagnetic interference, providing a reliable data foundation for autonomous navigation. The adaptive path planning algorithm improves the path search speed by 50% in complex environments and shortens the average path length by 20%, effectively reducing the UAV's flight energy consumption and improving operational efficiency. Simultaneously, optimization strategies for different scenarios enable the UAV to plan the optimal path in various environments. The fast-response obstacle avoidance algorithm reduces the response time to suddenly appearing obstacles to less than 0.1 seconds, and achieves a detection and avoidance success rate of over 95% for small obstacles (diameter less than 10 cm), greatly improving the safety of UAV flight and reducing the collision accident rate.

[0108] 7. The UAV autonomous path navigation system and its algorithm based on multimodal fusion and adaptive optimization described in this invention have wide applicability and are applicable to multiple fields such as forest fire monitoring, urban logistics distribution, agricultural plant protection, and power line inspection. They can meet the autonomous navigation needs of UAVs in different scenarios, have significant economic and social benefits, and are worthy of vigorous promotion and application. Attached Figure Description

[0109] Figure 1 This is a block diagram of the structure of the UAV autonomous path navigation system based on multimodal fusion and adaptive optimization described in this invention.

[0110] Figure 2 This is a schematic diagram of the UAV autonomous path navigation algorithm based on multimodal fusion and adaptive optimization described in this invention. Detailed Implementation

[0111] To better understand the principles and features of this invention, the appendix to the specification is now provided. Figures 1-2 This invention provides a detailed description of the specific implementation methods of the UAV autonomous path navigation system and its algorithm based on multimodal fusion and adaptive optimization.

[0112] Example

[0113] like Figure 1 As shown, the UAV autonomous path navigation system based on multimodal fusion and adaptive optimization of the present invention includes:

[0114] It consists of a multimodal sensor module, a data fusion processing module, an adaptive algorithm processing module, and a flight control module, among which:

[0115] The multimodal sensor module is sequentially and telecommunicationly connected to the data fusion processing module, adaptive algorithm processing module, and flight control module. The multimodal sensor module integrates a visual camera, lidar, millimeter-wave radar, ultrasonic sensor, and inertial measurement unit (IMU). The visual camera employs high dynamic range imaging technology, enabling it to clearly capture images under different lighting conditions, providing high-quality data for subsequent image feature extraction. The lidar uses a solid-state hybrid lidar, capable of quickly and accurately acquiring 3D point cloud data of the surrounding environment, precisely depicting the position and contour of obstacles. The millimeter-wave radar possesses high-resolution detection capabilities, enabling real-time monitoring of obstacle speed and orientation information, particularly suitable for detecting dynamic obstacles. The ultrasonic sensor is used for detecting nearby obstacles, assisting the visual camera, lidar, and millimeter-wave radar in achieving comprehensive perception of the UAV's surrounding environment. The inertial measurement unit measures the UAV's linear acceleration, angular velocity, and other motion information in real time, providing fundamental data for attitude calculation and position estimation.

[0116] Data fusion processing module: Utilizing a deep learning-based multimodal data fusion algorithm, this module preprocesses and fuses data collected from various sensors. It receives raw data from the multimodal sensor modules, first performing preprocessing operations such as filtering and noise reduction to remove noise interference and improve data quality. Then, the preprocessed data is input into a fusion network based on the Transformer architecture, where an attention mechanism automatically assigns weights to each modality of data, achieving deep data fusion.

[0117] The adaptive algorithm processing module includes adaptive optimization algorithms such as environment modeling, path planning, and obstacle avoidance. The adaptive algorithm processing module receives fused data from the data fusion processing module, performs environment modeling, path planning, and obstacle avoidance processing respectively, and dynamically adjusts the algorithm parameters according to the environment model and flight mission requirements to adapt to different environments and mission needs.

[0118] The flight control module, based on the output of the adaptive algorithm processing module, precisely controls the drone's flight attitude and speed.

[0119] The flight control module receives flight commands generated by path planning and obstacle avoidance algorithms, and achieves stable flight and autonomous navigation of the UAV by controlling the motor speed, servo angle and other actuators.

[0120] In one specific embodiment of the present invention, the multimodal sensor module collects images, point clouds, speed, and orientation data of the environment surrounding the UAV, as well as the motion information of the UAV itself, according to the sampling frequency: visual camera: 30-60 FPS, lidar: 10-20 Hz, millimeter-wave radar: 20-50 Hz, ultrasonic sensor: 50-100 Hz, and transmits these raw data to the data fusion processing module.

[0121] Example 2

[0122] like Figure 2 As shown, the UAV autonomous path navigation algorithm based on multimodal fusion and adaptive optimization described in this invention includes:

[0123] Step 1, data acquisition: Using a multimodal sensor module, images, point clouds, speed, azimuth, and other data of the environment around the UAV are collected simultaneously at a certain sampling frequency, as well as the motion information of the UAV itself. These raw data are then transmitted to the data fusion processing module.

[0124] Step 2, data fusion: The data fusion processing module preprocesses the collected raw data to remove noise and outliers, and then inputs the preprocessed data into a fusion network based on the Transformer architecture. The weights of each modality are calculated through the attention mechanism, and the data of different modalities are deeply fused to obtain fused data containing rich environmental information, and then transmitted to the adaptive algorithm processing module.

[0125] Step 3: Adaptive Algorithm Processing. The adaptive algorithm processing module performs multimodal environment modeling, path planning, and obstacle avoidance processing based on the fused data. The multimodal fusion environment modeling algorithm uses the fused data to construct a 3D environment model, updates the state of each voxel in the model in real time, and distinguishes between passable and impassable areas. Based on the target location information and the environment model, the adaptive path planning algorithm generates the optimal flight path. The obstacle avoidance algorithm monitors obstacles in the environment in real time. When an obstacle is detected, an appropriate obstacle avoidance strategy is adopted according to the threat level of the obstacle, and an obstacle avoidance path is generated. Among them, multimodal refers to the fusion mode of multiple different types of sensor data, including visual modality: image / video data from visual cameras containing high dynamic range imaging information; point cloud modality: 3D environmental point cloud data from LiDAR, used to construct spatial obstacle contours; radar modality: target velocity and azimuth information from millimeter-wave radar, used for dynamic obstacle detection; sonar modality: near-range ranging data from ultrasonic sensors, assisting in near-field obstacle perception; and inertial modality: attitude information from the UAV itself, including motion state data such as acceleration and angular velocity from the IMU.

[0126] Step 4: UAV flight control. The flight control module receives flight commands generated by path planning and obstacle avoidance algorithms. By controlling the UAV's actuators, it adjusts the UAV's flight attitude and speed, enabling the UAV to fly along the planned path. At the same time, it continuously provides feedback on the UAV's actual flight status, forming a closed-loop control system to ensure the UAV's flight stability and safety.

[0127] As a specific embodiment of the present invention, in step 3, the multimodal fusion environment modeling algorithm uses deep learning-based multimodal data fusion technology for environment modeling:

[0128] First, visual feature extraction is performed on sensor data from different modalities to obtain information such as the shape, texture, distance, and speed of obstacles; visual image feature extraction:

[0129] Convolutional Neural Networks (CNNs) are used to extract features from visual images to obtain information such as the shape and texture of obstacles. Let the input visual image be I∈R. H*W*CWhere H, W, and C represent the height, width, and number of channels of the image, respectively, the feature representation of the visual image is obtained through a series of convolutional layers, pooling layers, and activation functions: F v ∈R h*w*c , where h, w, and c are the height, width, and number of channels of the feature map, respectively;

[0130] Then, these feature information are input into a fusion network based on the Transformer architecture, and the weights of each modality data are automatically assigned through the attention mechanism to achieve deep data fusion;

[0131] Finally, the voxel octree algorithm is used to construct a three-dimensional environment model. The state of each voxel in the model is updated in real time according to the fused data to distinguish between passable and impassable areas.

[0132] LiDAR point cloud feature extraction: LiDAR data is processed using a point cloud processing algorithm to obtain the precise distance and 3D contour of obstacles. Let the input LiDAR point cloud data be P = {p1, p2, ..., p...} n}, where p i =(x i ,y i ,z i Let F be the 3D coordinates of the i-th point. A deep learning-based point cloud feature extraction network (such as PointNet or PointNet++) is used to convert the point cloud data into a feature vector F. l ∈R d , where d is the dimension of the feature vector;

[0133] Millimeter-wave radar data processing: Millimeter-wave radar data is converted into the speed and orientation information of obstacles through signal processing algorithms. Let the input millimeter-wave radar data be R = {r1, r2, ..., r...} m}, where r j =(v j ,θ j Let f(j) represent the velocity and orientation information of the j-th obstacle. The millimeter-wave radar data is then converted into a feature vector F using the feature mapping function fR. m ∈R k , where k is the dimension of the feature vector;

[0134] Ultrasonic sensor data processing: The ultrasonic sensor measures proximity information of surrounding objects in real time. Let the measurement data be U = {u1, u2, ..., u...} q}, u i Let represent the distance value of the i-th measurement. First, process it using the Kalman filter algorithm. The prediction equation of the Kalman filter is as follows (1):

[0135]

[0136] P k|k-1 =AP k-1|k-1 A T +Q......(1),

[0137] The updated equation is as follows (2):

[0138] K k =P k|k-1 H T HP k|k-1 H T +R) -1

[0139]

[0140] P k|k =(IK k H)P k|k-1 ......(2),

[0141] in, and These are the predicted state and the updated state, respectively; P k∣k-1 and P k∣k A is the corresponding covariance matrix; B is the state transition matrix; C is the control matrix. H is the control variable; H is the observation matrix; Q is the process noise covariance matrix; R is the observation noise covariance matrix; K k Kalman gain;

[0142] After filtering, a stable near-range feature vector is obtained: F u ∈R l l is the dimension of the feature vector;

[0143] Inertial Measurement Unit (IMU) Data Processing: The accelerometer and gyroscope data acquired by the IMU are respectively A acc ={a x ,a y ,a z} and A gyro ={ω x ,ω y ,ω z The complementary filtering algorithm is used to fuse the two data to eliminate drift error. The complementary filtering is as follows (3):

[0144]

[0145] θ z =θ z +ω2Δt......(3)

[0146] Where θx, θy, θz are the fused attitude angles, α is the complementary filter coefficient, and Δt is the sampling time interval. After processing, the accurate attitude and motion feature vectors of the UAV are obtained: F imu ∈R m m is the dimension of the feature vector;

[0147] As a specific embodiment of the present invention, in step 3, the multimodal data fusion involves combining the extracted visual image features F v LiDAR point cloud features F l Millimeter-wave radar characteristics F m Ultrasonic sensor characteristics F u and IMU features F imu The input is fed into a Transformer-based fusion network, which consists of multiple encoder layers. Each encoder layer includes a multi-head self-attention mechanism and a feedforward neural network. Through the multi-head self-attention mechanism, the network can automatically learn the correlations and weights between the data from different modalities. Let the fused feature vector be F. f ∈R D Where D is the dimension of the fused feature vector, and the fusion process is represented as: F f =Transformer(F v ,F l ,F m ,F u ,F imu );

[0148] As a specific embodiment of the present invention, in step 3, the environment modeling involves: constructing a three-dimensional environment model using a voxel octree algorithm, dividing the space around the UAV into multiple voxels, each voxel having a state value indicating whether there are obstacles within that voxel, and based on the fused feature vector F... f The classifier determines the state of each voxel, where s∈{0,1}, where 0 represents a passable region and 1 represents an impassable region. Simultaneously, by combining the UAV's attitude and position information provided by the IMU, the model accurately determines the UAV's own position and attitude. The classification process can be represented as: s=classifier(F f The model updates the drone's position and attitude in the model based on IMU data.

[0149] As a specific embodiment of the present invention, in step 3, the adaptive path planning algorithm introduces adaptive weight adjustment and scene optimization strategies on the basis of the traditional A* algorithm. Through online learning algorithm, the weight coefficients in the cost function are dynamically adjusted according to the environmental complexity and the flight status of the UAV, so that the algorithm can balance between safety and efficiency. At the same time, special heuristic functions are designed for different scenarios to guide the algorithm to plan a better path.

[0150] The traditional dynamic cost function of the A* algorithm is F(n) = G(n) + H(n), where G(n) is the actual cost from the starting point to node n, calculated based on physical distance, flight time, or energy consumption; H(n) is the estimated cost from node n to the target point, using a scene adaptive function; and C(n) is the environmental complexity cost, calculated based on obstacle density, dynamic obstacle speed, etc. In this algorithm, an adaptive weight adjustment strategy is introduced, and the cost function becomes F(n) = αG(n) + βH(n) + γC(n), where α, β, and γ are weight coefficients, and α + β + γ = 1.

[0151] Dynamic weight adjustment model: This model employs a reinforcement learning framework for weight optimization. By constructing a state space, action space, and reward function, and utilizing the Q-learning algorithm, it dynamically adjusts the weight coefficients, enabling the UAV to adaptively optimize its path planning strategy under different environments and mission conditions.

[0152] The detailed definition and calculation of the state space S: The state space S describes the environment in which the UAV is located and its own state information. It consists of two main parts: environmental complexity indicators and UAV state indicators.

[0153] Environmental complexity metric: Obstacle density. The space surrounding the drone is divided into multiple identical cubic cells. The volume ratio occupied by obstacles in each cell is calculated, and then the average ratio across all cells is taken to obtain the obstacle density. Let there be N cells in total, and the obstacle volume in the i-th cell be V. i The total volume of the cells is V. total Then the obstacle density D obs The calculation is as follows (4):

[0154]

[0155] Equation (4) above shows that the higher the obstacle density, the more complex the environment, and the higher the safety requirements for path planning;

[0156] Distribution entropy measures the uniformity of obstacle distribution in space. Based on the concept of Shannon entropy, let p be the probability of an obstacle appearing in different regions. j Let j represent different regions, then the distribution entropy is as follows (5):

[0157]

[0158] Equation (5) above shows that the greater the distribution entropy, the more dispersed and irregular the distribution of obstacles, and the more difficult the path planning becomes.

[0159] The drone status indicators include:

[0160] Drone speed, the current flight speed of a drone, includes three components, namely v x v y v z Combine them into a velocity vector The magnitude and direction of speed affect the efficiency and feasibility of path planning. For example, high-speed flight requires more space for obstacle avoidance and demands a smoother path.

[0161] Drone battery level: E, representing the remaining percentage of the drone's battery charge. rem This means that it directly relates to the time the drone can continue flying and the range of tasks it can perform. When the battery is low, path planning needs to prioritize the shortest path to complete the task or return to the charging point as quickly as possible.

[0162] Remaining task time: Calculated based on the total task time requirement and the time already spent executing. Let the total task time be T. total The execution time is T. exec The remaining time for the task is: T rem =T total -T exec When the remaining time for a task is tight, the algorithm needs to choose the most efficient path possible while ensuring safety.

[0163] Integrating all the above indicators together forms the state vector: S = [D obs E dis ,v x ,v y ,v z E rem ,T rem This comprehensively reflects the current environment and status of the drone;

[0164] The specific contents and constraints of action space A:

[0165] The action space A is defined as a weight coefficient vector [α,β,γ], which is used to adjust the path planning cost function.

[0166] The proportions of each component in F(n)=α·G(n)+β·H(n)+γ·C(n), where:

[0167] α controls the actual cost G from the starting point to node n. The weight (n) of G(n) is usually calculated based on physical distance, flight time or energy consumption. When α is large, the algorithm will be more inclined to choose the path with lower actual cost and focus on the economic efficiency of the path.

[0168] β controls the weight of the heuristic estimation cost H(n) from node n to the target. H(n) adopts a scene adaptive function. Increasing the value of β can speed up the search speed of the algorithm and make the UAV move towards the target faster, but may sacrifice some safety.

[0169] γ controls the weight of the environmental complexity cost C(n), which is calculated based on obstacle density, dynamic obstacle speed, etc. Increasing γ allows the algorithm to focus more on avoiding dangerous areas in complex environments, ensuring flight safety.

[0170] To ensure the rationality of the cost function, the weight coefficients satisfy α + β + γ = 1, and α ≥ 0, β ≥ 0, γ ≥ 0. In practical applications, UAVs adjust the values ​​of [α, β, γ] to change the focus of path planning, adapting to different environments and mission requirements.

[0171] Multidimensional calculation methods for the reward function R:

[0172] The reward function R is used to evaluate the effect of the UAV's state change after performing the action. It comprehensively considers multiple dimensions such as path length, flight time, energy consumption, and safety margin. Its calculation formula is as follows (6):

[0173] R=w1·ΔL+w2·ΔT+w3·ΔE+w4·ΔS

[0174] Where: w1, w2, w3, and w4 are the weight coefficients of each dimension indicator, set according to the priority and importance of the task, and w1 + w2 + w3 + w4 = 1; ΔL represents the change in path length, which is the current path length L. cur Compared to the previous path length L pre The difference, i.e., ΔL = L cur -L pre Shortening the path length helps improve task execution efficiency, so a positive reward is given when ΔL < 0; otherwise, a negative reward is given.

[0175] ΔT represents the change in flight time, and is calculated similarly to path length: ΔT = T cur -T pre Reducing flight time allows for faster mission completion; similarly, a positive reward is obtained when ΔT < 0.

[0176] ΔE represents the change in energy consumption, ΔE = E cur -E preReducing energy consumption can extend the drone's endurance, and a positive reward is given when ΔE < 0.

[0177] ΔS represents the change in safety margin, which is assessed by calculating factors such as the distance between the drone and obstacles and hazardous areas. Let S be the safety margin index. index Then ΔS = S index (cur)-S index (pre). When the safety margin increases, ΔS>0, and a positive reward is given to encourage the drone to choose a safer path;

[0178] The iterative update process of the Q-learning algorithm;

[0179] The Q-learning algorithm iteratively updates the optimal strategy through continuous trial and error and learning, and its expression is as follows (6):

[0180]

[0181] Where: Q(s,a) represents the Q value of performing action a in state s, which represents the expected cumulative reward of the action in the current state; η is the learning rate, which ranges from [0,1] and is used to control the degree of adoption of new information in each update. The closer η is to 1, the more the algorithm focuses on the current reward, and the faster the learning speed is but may not be stable; the closer η is to 0, the more the algorithm relies on previous experience, the smoother the learning process is but the slower the convergence speed is.

[0182] R(s,a) is the immediate reward obtained after performing action a in state s, which is calculated by the reward function;

[0183] γ is a discount factor, ranging from [0,1], used to measure the importance of future rewards. The closer γ is to 1, the more the algorithm values ​​future rewards and considers long-term gains; the closer γ is to 0, the more it focuses on immediate rewards.

[0184] s′ is the new state transitioned to after performing action a, and a′max Q(s′,a′) represents the maximum value of Q among all possible actions in the new state s′, which represents the optimal expected cumulative reward that can be obtained from the new state;

[0185] The specific iterative update process is as follows:

[0186] In the initial state s0, the drone selects action a0 according to the v-greedy policy. The ∈-greedy policy randomly selects actions with a probability of ∈ and selects the action with the largest current Q value with a probability of 1-∈. By adjusting the value of ∈, the relationship between exploring new actions and utilizing existing experience can be balanced.

[0187] After performing action a0, the drone transitions to a new state s1 and receives an immediate reward R(s0,a0). The value of Q(s0,a0) is then updated according to the Q-learning formula.

[0188] By repeating the above steps, continuously selecting actions, executing actions, obtaining rewards, and updating the Q value under different states, the Q value gradually converges as the number of iterations increases. The drone is able to learn the optimal action under various states, i.e., the optimal path planning strategy.

[0189] Through the detailed description of the state space, action space, reward function, and Q-learning algorithm above, the dynamic adjustment model of weight coefficients enables UAVs to autonomously learn and optimize the weight coefficients of path planning under complex environments and changing mission requirements, thereby achieving efficient and safe path navigation.

[0190] Design dedicated heuristic functions for different scenarios:

[0191] Urban canyon environment: as shown in equation (7):

[0192] H city (n)=w1·d euclid (n,g)+w2·h building (n)+w3·t shadouw (n)......(7),

[0193] Where: d euclid (n, g): Euclidean distance from node n to the target, h building (n): Average height of buildings surrounding node n, t shadow (n): Probability that node n is located in the shaded area of ​​a building; w1, w2, w3: Environmental adaptive weights, dynamically adjusted by an SVM classifier.

[0194] Open terrain environment: as shown in formula (8):

[0195] H open (n)=w1·d euclid (n,g)+w2·v wind (n)·d euclid (n,g)......(8),

[0196] Where: v wind (n) represents the estimated wind speed at node n, which is obtained by fusing meteorological data and sensor data.

[0197] Complex indoor environments: as shown in equation (9):

[0198]

[0199] Where: dobs (n,i): The distance from node n to the i-th obstacle, and k represents the number of obstacles within the field of view.

[0200] The specific algorithm implementation of this invention is as follows:

[0201] def adaptive_path_planning(start,goal,environment_model,drone_state):

[0202] #Initialize the open and closed lists

[0203] open_set = PriorityQueue()

[0204] closed_set = set()

[0205] #Initialize the starting node

[0206] start_node=Node(start,g=0,h=heuristic_evaluation(start,goal,environment_model),c=complexity_evaluation(start,environment_model))

[0207] start_node.f=calculate_adaptive_f(start_node,drone_state)

[0208] open_set.put((start_node.f,start_node))

[0209] which does not open_set.empty():

[0210] #Find the node with the smallest f value

[0211] current_node=open_set.get()[1]

[0212] # Reach the target node and return the path

[0213] if current_node.position==goal:

[0214] return reconstruct_path(current_node)

[0215] closed_set.add(current_node.position)

[0216] # Generate adjacent nodes

[0217] neighbors=generate_neighbors(current_node,environment_model,drone_state)

[0218] for neighbor in neighbors:

[0219] #Check if there is an obstacle or if it has already been visited.

[0220] if neighbor.position in closed_set or is_obstacle(neighbor.position,environment_model):

[0221] Continue

[0222] #Calculate the temporary g value

[0223] temp_g=current_node.g+movement_cost(current_node,neighbor,environment_model,drone_state)

[0224] #If the neighbor is not on the open list or the new path is better

[0225] if not is_in_open_set(neighbor,open_set)or temp_g <neighbor.g:

[0226] neighbor.parent=current_node

[0227] neighbor.g = temp_g

[0228] neighbor.h=heuristic_evaluation(neighbor.position,goal,environment_model)

[0229] neighbor.c=complexity_evaluation(neighbor.position,environment_model)

[0230] neighbor.f=calculate_adaptive_f(neighbor,drone_state)

[0231] if not is_in_open_set(neighbor,open_set):

[0232] open_set.put((neighbor.f,neighbor))

[0233] #Path not found

[0234] return None

[0235] def calculate_adaptive_f(node,drone_state):

[0236] #Dynamically calculate weighting coefficients based on UAV status

[0237] alpha,beta,gamma=update_weights(drone_state)

[0238] #Calculate the adaptive f value

[0239] return alpha*node.g+beta*node.h+gamma*node.c

[0240] def update_weights(drone_state):

[0241] #Get current environment and drone status parameters

[0242] obstacle_density=drone_state.environment.obstacle_density

[0243] remaining_battery=drone_state.battery_level

[0244] mission_urgency=drone_state.mission_priority

[0245] # Calculate weights for the network using a reinforcement learning strategy

[0246] alpha=policy_network.predict([obstacle_density,remaining_battery,mission_urgency])[0]

[0247] beta=policy_network.predict([obstacle_density,remaining_battery,mission_urgency])[1]

[0248] gamma=1-alpha-beta return alpha,beta,gamma

[0249] As a specific embodiment of the present invention, in step 3, the fast response obstacle avoidance algorithm is as follows:

[0250] Rapid obstacle avoidance is achieved based on an improved Dijkstra algorithm and a dynamic risk assessment mechanism. When the multimodal sensor detects an obstacle, the threat level of the obstacle to the UAV is first calculated using a dynamic risk assessment algorithm. For high-threat obstacles, a bidirectional search Dijkstra algorithm is used to quickly generate an obstacle avoidance path; for low-threat obstacles, a local path adjustment strategy is adopted to fine-tune the original path, reducing the computational load of path planning. Simultaneously, reinforcement learning algorithms are used to optimize the obstacle avoidance strategy, enabling the UAV to autonomously select the optimal obstacle avoidance method in different scenarios. The specific steps and calculation formulas are as follows:

[0251] Dynamic risk assessment: Taking into account factors such as the obstacle's speed, distance, and size, as well as the drone's flight speed and direction, the threat level R of the obstacle to the drone is calculated. Let the obstacle's speed be v. o The distance is d o Size is s o The drone's flight speed is v u The direction is θ u The formula for calculating the threat level is as follows (10):

[0252]

[0253] Where w1, w2, w3, and w4 are weighting coefficients, and θ O The direction of the obstacle's movement;

[0254] Obstacle avoidance strategy selection: For high-threat level obstacles, when R>R th (R thWhen the threat level threshold is reached, the Dijkstra algorithm with bidirectional search is adopted. The search start and end points are the current position S of the UAV and the nearest unreachable node T in the original path plan. The search starts from both S and T. When the two search paths meet, the obstacle avoidance path is obtained.

[0255] Low threat level obstacles: when R≤R th In this case, a local path adjustment strategy is adopted. Based on the original path, the path is finely adjusted according to the position and direction of movement of the obstacles, so that the drone can avoid the obstacles.

[0256] Reinforcement Learning Optimization: The obstacle avoidance strategy is optimized using reinforcement learning algorithms. The obstacle avoidance process of the UAV is regarded as a Markov Decision Process (MDP), defining the state space S, action space A, and reward function R. Through continuous simulation training, the UAV learns the optimal obstacle avoidance strategy. Let the state be s∈S and the action be a∈A, then the goal of reinforcement learning is to maximize the cumulative reward as shown in equation (11):

[0257]

[0258] Where π is the strategy, γ is the discount factor, and T is the number of time steps.

[0259] This invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the invention as claimed. The scope of protection of this invention is defined by the appended claims.

Claims

1. An autonomous path navigation system for unmanned aerial vehicles (UAVs) based on multimodal fusion and adaptive optimization, characterized in that, include: The system comprises a multimodal sensor module, a data fusion processing module, an adaptive algorithm processing module, and a flight control module. The multimodal sensor module is sequentially and electrically connected to the data fusion processing module, the adaptive algorithm processing module, and the flight control module. The multimodal sensor module integrates a visual camera, LiDAR, millimeter-wave radar, ultrasonic sensor, and inertial measurement unit. The visual camera employs a high dynamic range imaging technology of 100-140dB, enabling it to clearly capture images under different lighting conditions, providing high-quality data for subsequent image feature extraction. The LiDAR uses a solid-state hybrid LiDAR to acquire 3D point cloud data of the surrounding environment, depicting the position and outline of obstacles. The millimeter-wave radar has high-resolution detection capabilities, enabling real-time monitoring of obstacle speed and orientation information, and is particularly suitable for detecting dynamic obstacles. The ultrasonic sensor is used for detecting nearby obstacles, assisting the visual camera, LiDAR, and millimeter-wave radar in achieving comprehensive perception of the UAV's surrounding environment. The inertial measurement unit measures the UAV's linear acceleration, angular velocity, and other motion information in real time, providing basic data for attitude calculation and position estimation. The data fusion processing module uses a deep learning-based multimodal data fusion algorithm to preprocess and fuse the data collected by each sensor. The data fusion processing module receives the raw data from the multimodal sensor module, performs preprocessing operations such as filtering and noise reduction on the data to remove noise interference, and inputs the preprocessed data into the fusion network based on the Transformer architecture. The attention mechanism automatically allocates the weights of each modality of data to achieve deep data fusion. The adaptive algorithm processing module includes adaptive optimization algorithms such as environment modeling, path planning, and obstacle avoidance. The adaptive algorithm processing module receives fused data from the data fusion processing module, performs environment modeling, path planning, and obstacle avoidance processing respectively, and dynamically adjusts the algorithm parameters according to the environment model and flight mission requirements to adapt to different environments and mission needs. The flight control module, based on the output of the adaptive algorithm processing module, precisely controls the drone's flight attitude and speed. The flight control module receives flight commands generated by path planning and obstacle avoidance algorithms, and achieves stable flight and autonomous navigation of the UAV by controlling the motor speed, servo angle and other actuators.

2. The UAV autonomous path navigation system based on multimodal fusion and adaptive optimization as described in claim 1, characterized in that, The multimodal sensor module collects images, point clouds, speed, and orientation data of the environment surrounding the UAV, as well as the UAV's own motion information, according to the sampling frequency: visual camera: 30-60 FPS, lidar: 10-20 Hz, millimeter-wave radar: 20-50 Hz, and ultrasonic sensor: 50-100 Hz. It then transmits these raw data to the data fusion processing module.

3. The algorithm for an autonomous path navigation system for unmanned aerial vehicles based on multimodal fusion and adaptive optimization as described in claim 1, comprising the following steps: Step 1, data acquisition: Using a multimodal sensor module, images, point clouds, speed, azimuth, and other data of the environment around the UAV are collected simultaneously at a certain sampling frequency, as well as the motion information of the UAV itself. These raw data are then transmitted to the data fusion processing module. Step 2, data fusion: The data fusion processing module preprocesses the collected raw data to remove noise and outliers, and then inputs the preprocessed data into a fusion network based on the Transformer architecture. The weights of each modality are calculated through the attention mechanism, and the data of different modalities are deeply fused to obtain fused data containing rich environmental information, and then transmitted to the adaptive algorithm processing module. Step 3, Algorithm Processing: The adaptive algorithm processing module performs multimodal environment modeling, path planning, and obstacle avoidance based on the fused data. The multimodal environment modeling algorithm: constructs a 3D environment model using the fused data, updates the state of each voxel in the model in real time, and distinguishes between passable and impassable areas; based on the target location information and the environment model, it uses an adaptive path planning algorithm to generate the optimal flight path; the obstacle avoidance algorithm monitors obstacles in the environment in real time, and when an obstacle is detected, it adopts an appropriate obstacle avoidance strategy based on the obstacle's threat level, generating an obstacle avoidance path. Specifically: multimodal refers to the fusion mode of various types of sensor data, including: visual modality: image / video data from a visual camera containing high dynamic range imaging information; point cloud modality: 3D environmental point cloud data from LiDAR, used to construct spatial obstacle outlines; radar modality: target velocity and azimuth information from millimeter-wave radar, used for dynamic obstacle detection; sonar modality: near-range ranging data from ultrasonic sensors, assisting in near-field obstacle perception; and inertial modality: attitude information from the UAV itself, including acceleration, angular velocity, and other motion state data from the IMU. Step 4: UAV flight control. The flight control module receives flight commands generated by path planning and obstacle avoidance algorithms. By controlling the UAV's actuators, it adjusts the UAV's flight attitude and speed, enabling the UAV to fly along the planned path. At the same time, it continuously provides feedback on the UAV's actual flight status, forming a closed-loop control system to ensure the UAV's flight stability and safety.

4. In the algorithm for the UAV autonomous path navigation system based on multimodal fusion and adaptive optimization according to claim 3, in step 3, the multimodal fusion environment modeling algorithm uses deep learning-based multimodal data fusion technology for environment modeling: First, visual feature extraction is performed on sensor data from different modalities to obtain information such as the shape, texture, distance, and speed of obstacles; visual image feature extraction: Convolutional neural networks are used to extract features from visual images to obtain information such as the shape and texture of obstacles. Let the input visual image be I∈R. H*W*C Where H, W, and C represent the height, width, and number of channels of the image, respectively, the feature representation of the visual image is obtained through a series of convolutional layers, pooling layers, and activation functions: F v ∈R h*w*c , where h, w, and c are the height, width, and number of channels of the feature map, respectively; Then, these feature information are input into a fusion network based on the Transformer architecture, and the weights of each modality data are automatically assigned through the attention mechanism to achieve deep data fusion; Finally, the voxel octree algorithm is used to construct a three-dimensional environment model. The state of each voxel in the model is updated in real time according to the fused data to distinguish between passable and impassable areas. LiDAR point cloud feature extraction: LiDAR data is processed using a point cloud processing algorithm to obtain the precise distance and 3D contour of obstacles. Let the input LiDAR point cloud data be P = {p1, p2, ..., p...} n }, where p i =(x i ,y i ,z i Let F be the 3D coordinates of the i-th point. A deep learning-based point cloud feature extraction network, such as PointNet or PointNet++, is used to convert the point cloud data into a feature vector F. l ∈R d , where d is the dimension of the feature vector; Millimeter-wave radar data processing: Millimeter-wave radar data is converted into the speed and orientation information of obstacles through signal processing algorithms. Let the input millimeter-wave radar data be R = {r1, r2, ..., r...} m }, where r j =(v j ,θ j Let f(j) represent the velocity and orientation information of the j-th obstacle. The millimeter-wave radar data is then converted into a feature vector F using the feature mapping function fR. m ∈R k , where k is the dimension of the feature vector; Ultrasonic sensor data processing: The ultrasonic sensor measures proximity information of surrounding objects in real time. Let the measurement data be U = {u1, u2, ..., u...} q }, u i Let represent the distance value of the i-th measurement. First, process it using the Kalman filter algorithm. The prediction equation of the Kalman filter is as follows (1): P k|k-1 =AP k-1k-1 From T +Q......(1), The updated equation is as follows (2): K k =P k|k-1 H T (HP k|k-1 H T +R) -1 P k|k =(I-K k H)P k|k-1 ......(2), in, and These are the predicted state and the updated state, respectively; P k∣k-1 and P k∣k This is the corresponding covariance matrix; A is the state transition matrix; B is the control matrix; H is the control variable; H is the observation matrix; Q is the process noise covariance matrix; R is the observation noise covariance matrix; K k Kalman gain; After filtering, a stable near-range feature vector is obtained: F u ∈R l l is the dimension of the feature vector; Inertial Measurement Unit (IMU) Data Processing: The accelerometer and gyroscope data acquired by the IMU are respectively A acc ={a x ,a y ,a z } and A gyro ={ω x ,ω y ,ω z The complementary filtering algorithm is used to fuse the two data to eliminate drift error. The complementary filtering is as follows (3): i z =θ z +oh z △t......(3), Where θx, θy, θz are the fused attitude angles, α is the complementary filter coefficient, and Δt is the sampling time interval. After processing, the accurate attitude and motion feature vectors of the UAV are obtained: F imu ∈R m , where m is the dimension of the feature vector.

5. The algorithm for an autonomous path navigation system for unmanned aerial vehicles based on multimodal fusion and adaptive optimization according to claim 3, in step 2, the multimodal data fusion involves: combining the extracted visual image features F... v LiDAR point cloud features F l Millimeter-wave radar characteristics F m Ultrasonic sensor characteristics F u and IMU features F imu The input is fed into a Transformer-based fusion network, which consists of multiple encoder layers. Each encoder layer includes a multi-head self-attention mechanism and a feedforward neural network. Through the multi-head self-attention mechanism, the network can automatically learn the correlations and weights between the data from different modalities. Let the fused feature vector be F. f ∈R D ,in, D is the dimension of the fused feature vector, and the fusion process is represented as: F f =Transformer, where the parameter of Transformer is F v ,F l ,F m ,F u ,F imu .

6. According to the algorithm of the UAV autonomous path navigation system based on multimodal fusion and adaptive optimization as described in claim 3, in step 3, the adaptive environment model modeling is as follows: a three-dimensional environment model is constructed using the voxel octree algorithm, dividing the space around the UAV into multiple voxels, each voxel having a state value indicating whether there are obstacles within that voxel, and based on the fused feature vector F... f The classifier determines the state of each voxel, where s∈{0,1}, where 0 represents a passable region and 1 represents an impassable region. Simultaneously, by combining the UAV's attitude and position information provided by the IMU, the model accurately determines the UAV's own position and attitude. The classification process can be represented as: s=classifier(F f The model updates the drone's position and attitude in the model based on IMU data.

7. The algorithm for the UAV autonomous path navigation system based on multimodal fusion and adaptive optimization as described in claim 3, wherein in step 3, the adaptive path planning algorithm, based on the traditional A* algorithm, introduces adaptive weight adjustment and scene optimization strategies, and dynamically adjusts the weight coefficients in the cost function according to the environmental complexity and the flight state of the UAV through an online learning algorithm, so that the algorithm can balance between safety and efficiency. At the same time, a special heuristic function is designed for different scenarios to guide the algorithm to plan a better path; The traditional dynamic cost function of the A* algorithm is F(n) = G(n) + H(n), where G(n) is the actual cost from the starting point to node n, calculated based on physical distance, flight time, or energy consumption; H(n) is the estimated cost from node n to the target point, using a scene adaptive function; and C(n) is the environmental complexity cost, calculated based on obstacle density, dynamic obstacle speed, etc. In this algorithm, an adaptive weight adjustment strategy is introduced, and the cost function becomes F(n) = αG(n) + βH(n) + γC(n), where α, β, and γ are weight coefficients, and α + β + γ = 1. Dynamic weight adjustment model: This model employs a reinforcement learning framework for weight optimization. By constructing a state space, action space, and reward function, and utilizing the Q-learning algorithm, it dynamically adjusts the weight coefficients, enabling the UAV to adaptively optimize its path planning strategy under different environments and mission conditions. The detailed definition and calculation of the state space S: The state space S describes the environment in which the UAV is located and its own state information. It consists of two main parts: environmental complexity indicators and UAV state indicators. Environmental complexity metric: Obstacle density. The space surrounding the drone is divided into multiple identical cubic cells. The volume ratio occupied by obstacles in each cell is calculated, and then the average ratio across all cells is taken to obtain the obstacle density. Let there be N cells in total, and the obstacle volume in the i-th cell be V. i The total volume of the cells is V. total Then the obstacle density D obs The calculation is as follows (4): Equation (4) above shows that the higher the obstacle density, the more complex the environment, and the higher the safety requirements for path planning; Distribution entropy measures the uniformity of obstacle distribution in space. Based on the concept of Shannon entropy, let p be the probability of an obstacle appearing in different regions. j Let j represent different regions, then the distribution entropy is as follows (5): Equation (5) above shows that the greater the distribution entropy, the more dispersed and irregular the distribution of obstacles, and the more difficult the path planning becomes. The drone status indicators include: Drone speed, the current flight speed of a drone, includes three components, namely v x v y v z Combine them into a velocity vector Drone battery level: E represents the remaining percentage of the drone's battery charge. rem It indicates that when the battery is low, route planning should prioritize the shortest path to complete the task or return to the charging point as quickly as possible. Remaining task time: Calculated based on the total task time requirement and the time already spent executing. Let the total task time be T. total The execution time is T. exec The remaining time for the task is: T rem =T total -T exec When time is running out for a task, the algorithm needs to choose a more efficient path while ensuring safety. Integrating all the above indicators together forms the state vector: S = [D obs E dis ,v x ,v y ,v z E rem ,T rem This comprehensively reflects the current environment and status of the drone; The specific content and constraints of action space A: Action space A is defined as a weight coefficient vector [α, β, γ]. These weight coefficients are used to adjust the proportion of each part in the path planning cost function: F(n) = α·G(n) + β·H(n) + γ·C(n), where: α controls the actual cost G from the starting point to node n. The weight (n) of G(n) is usually calculated based on physical distance, flight time or energy consumption. When α is large, the algorithm will be more inclined to choose the path with lower actual cost and focus on the economic efficiency of the path. β controls the weight of the heuristic estimation cost H(n) from node n to the target. H(n) adopts a scene adaptive function. Increasing the value of β can speed up the search speed of the algorithm. γ controls the weight of the environmental complexity cost C(n), which is calculated based on obstacle density and dynamic obstacle speed. Increasing γ enables the algorithm to focus more on avoiding dangerous areas in complex environments, thus ensuring flight safety. To ensure the rationality of the cost function, the weight coefficients satisfy α+β+γ=1, and α≥0, β≥0, γ≥0. The UAV adjusts the values ​​of [α,β,γ] to change the focus of path planning to adapt to different environments and mission requirements. Multi-dimensional calculation method of reward function R: The reward function R is used to evaluate the state change effect of the UAV after performing the action. It comprehensively considers multiple dimensions such as path length, flight time, energy consumption, and safety margin. Its calculation formula is as follows (6): R=w1·ΔL+w2·ΔT+w3·ΔE+w4·ΔS Where: w1, w2, w3, and w4 are the weight coefficients of each dimension indicator, set according to the priority and importance of the task, and w1 + w2 + w3 + w4 = 1; ΔL represents the change in path length, which is the current path length L. cur Compared to the previous path length L pre The difference, i.e., ΔL = L cur -L pre Shortening the path length can improve task execution efficiency. When ΔL < 0, a positive reward will be given; otherwise, a negative reward will be given. ΔT represents the change in flight time, and is calculated similarly to path length: ΔT = T cur -T pre Reducing flight time allows for faster mission completion; similarly, a positive reward is obtained when ΔT < 0. ΔE represents the change in energy consumption, ΔE = E cur -E pre Reducing energy consumption can extend the drone's endurance, and a positive reward is given when ΔE < 0. ΔS represents the change in safety margin, which is assessed by calculating factors such as the distance between the drone and obstacles and hazardous areas. Let S be the safety margin index. index Then ΔS = S index (cur)-S index (pre). When the safety margin increases, ΔS>0, and a positive reward is given to encourage the drone to choose a safer path; The iterative update process of the Q-learning algorithm; The Q-learning algorithm iteratively updates the optimal strategy through continuous trial and error and learning, and its expression is as follows (6): Where: Q(s,a) represents the Q value of performing action a in state s, which represents the expected cumulative reward of the action in the current state; η is the learning rate, which ranges from [0,1] and is used to control the degree of adoption of new information in each update. The closer η is to 1, the more the algorithm focuses on the current reward, and the faster the learning speed is but may not be stable; the closer η is to 0, the more the algorithm relies on previous experience, the smoother the learning process is but the slower the convergence speed is. R(s,a) is the immediate reward obtained after performing action a in state s, which is calculated by the reward function; γ is a discount factor, which ranges from [0,1] and is used to measure the importance of future rewards. The closer γ is to 1, the more the algorithm values ​​future rewards and considers long-term benefits; the closer γ is to 0, the more it focuses on immediate rewards. s′ is the new state transitioned to after performing action a, and a′maxQ(s′,a′) represents the maximum Q value among all possible actions in the new state s′, which represents the optimal expected cumulative reward that can be obtained from the new state.

8. According to the algorithm of the UAV autonomous path navigation system based on multimodal fusion and adaptive optimization as described in claim 3, the specific iterative update procedure in step 3 is as follows: In the initial state s0, the drone selects action a0 according to the ∈-greedy policy. The ∈-greedy policy randomly selects actions with the probability of ∈ and selects the action with the largest current Q value with the probability of 1-∈. The relationship between exploring new actions and utilizing existing experience is balanced by adjusting the value of ∈. After performing action a0, the drone transitions to a new state s1 and receives an immediate reward R(s0,a0). The value of Q(s0,a0) is then updated according to the Q-learning formula. Repeat the above steps, continuously select actions, execute actions, obtain rewards and update Q value in different states. As the number of iterations increases, Q value gradually converges, and the drone can learn the optimal action in various states, that is, the optimal path planning strategy. Based on the above description of the state space, action space, reward function, and Q-learning algorithm, the dynamic adjustment model of weight coefficients enables UAVs to autonomously learn and optimize the weight coefficients of path planning under complex environments and changing mission requirements, thereby achieving efficient and safe path navigation.

9. The algorithm for the UAV autonomous path navigation system based on multimodal fusion and adaptive optimization as described in claim 7, wherein the scene optimization strategy is to design dedicated heuristic functions for different scenes: Urban canyon environment: as shown in equation (7): H city (n)=w1·d euclid (n,g)+w2·h building (n)+w3·t shadou (n)......(7), in: d euclid (n, g): Euclidean distance from node n to the target, h building (n): Average height of buildings surrounding node n, t shadow (n): Probability that node n is located in the shadow area of ​​a building; w1, w2, w3: Environmental adaptive weights, dynamically adjusted by an SVM classifier; Open terrain environment: as shown in formula (8): H open (n)=ω1·d euclid (n,g)+w2·v wind (n)·d euclid (n,g)......(8), Where: v wind(n) This represents the estimated wind speed at node n, obtained through the fusion of meteorological data and sensor data. Complex indoor environments: as shown in equation (9): Where: d obs (n,i): The distance from node n to the i-th obstacle, and k represents the number of obstacles within the field of view.

10. The algorithm for the UAV autonomous path navigation system based on multimodal fusion and adaptive optimization as described in claim 3, wherein in step 3, the obstacle avoidance algorithm is as follows: Rapid obstacle avoidance is achieved based on an improved Dijkstra algorithm and a dynamic risk assessment mechanism. When the multimodal sensor detects an obstacle, the threat level of the obstacle to the UAV is first calculated using a dynamic risk assessment algorithm. For high-threat obstacles, a bidirectional search Dijkstra algorithm is used to quickly generate an obstacle avoidance path. For low-threat obstacles, a local path adjustment strategy is adopted to fine-tune the original path, reducing the computational load of path planning. At the same time, a reinforcement learning algorithm is used to optimize the obstacle avoidance strategy, enabling the UAV to autonomously select the optimal obstacle avoidance method in different scenarios. The specific steps and calculation formulas are as follows: Dynamic risk assessment: Taking into account factors such as the obstacle's speed, distance, and size, as well as the drone's flight speed and direction, the threat level R of the obstacle to the drone is calculated. Let the obstacle's speed be v. o The distance is d o Size is s o The drone's flight speed is v u The direction is θ u The formula for calculating the threat level is as follows (10): in, w1, w2, w3, and w4 are weighting coefficients, and θ O The direction of the obstacle's movement; Obstacle avoidance strategy selection: For high-threat level obstacles, when R>R th (R th When the threat level threshold is reached, the Dijkstra algorithm with bidirectional search is adopted. The search starts and ends at the current position S of the UAV and the nearest unreachable node T in the original path plan. The search starts from both S and T. When the two search paths meet, the obstacle avoidance path is obtained. Low threat level obstacles: when R≤R th In this case, a local path adjustment strategy is adopted. Based on the original path, the path is finely adjusted according to the position and direction of movement of obstacles, so that the drone can avoid obstacles; Reinforcement Learning Optimization: The obstacle avoidance strategy is optimized using reinforcement learning algorithms. The obstacle avoidance process of the UAV is regarded as a Markov Decision Process (MDP), defining the state space S, action space A, and reward function R. Through continuous simulation training, the UAV learns the optimal obstacle avoidance strategy. Let the state be s∈S and the action be a∈A. Then the goal of reinforcement learning is to maximize the cumulative reward as shown in the following equation (11): Where π is the strategy, γ is the discount factor, and T is the number of time steps.

Images