A Method and System for Autonomous Obstacle Avoidance and Dynamic Path Planning of Unmanned Aerial Vehicles Based on Multimodal Perception and Hybrid Intelligent Decision Making

Abstract

This invention proposes a method and system for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles (UAVs) based on multimodal perception and hybrid intelligent decision-making. Environmental parameters such as wind speed, illuminance, temperature, and humidity, as well as image quality indicators, are collected in real time through an airborne weather station, IMU, and visual sensors. A flight parameter-image quality coupling model is established, and multi-objective optimization is achieved using Support Vector Regression (SVR) and Particle Swarm Optimization (PSO). A dynamic strategy optimization module is designed based on Q-learning, and a reward function is designed combining image quality, obstacle avoidance safety, and energy consumption. A 3D obstacle model is constructed in real time using ORB-SLAM3, dynamic hazard coefficients are calculated, and an improved APF-RRT* algorithm is used to generate obstacle avoidance trajectories. An online cooperative control module uses the BFGS algorithm to optimize control variables, ensuring flight stability and mission efficiency. This invention achieves high-precision obstacle avoidance and path planning for UAVs in complex environments, exhibiting strong robustness and wide adaptability, and is suitable for practical applications such as inspection and surveying.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation technology for unmanned aerial vehicles (UAVs), and in particular to a method and system for autonomous obstacle avoidance and dynamic path planning for UAVs based on multimodal perception and hybrid intelligent decision-making, which is applicable to autonomous navigation of UAVs in complex dynamic environments. Background Technology

[0002] With the rapid development of drone technology, its applications in fields such as inspection, surveying, and disaster relief are becoming increasingly widespread. However, autonomous flight of drones in complex environments still faces many challenges, including dynamic obstacle avoidance, poor environmental adaptability, and the difficulty in balancing flight efficiency and mission quality. Traditional drone obstacle avoidance methods mainly rely on a single sensor (such as vision or radar), which is difficult to cope with complex conditions such as changes in lighting and weather interference, and path planning algorithms often lack optimization considerations for mission requirements (such as imaging quality).

[0003] In existing technologies, while vision-based SLAM (such as the ORB-SLAM series) can construct environmental maps, it does not fully integrate multimodal environmental data (such as wind speed and illumination) for flight decisions; path planning algorithms (such as RRT* and APF) mostly focus on static obstacles and lack real-time response capabilities to dynamic obstacles; and control optimization methods (such as PID and MPC) are usually independent of the perception module, resulting in poor system coordination. Furthermore, existing solutions lack systematic optimization in balancing image quality, obstacle avoidance safety, and energy consumption, limiting the practicality of UAVs in long-term missions.

[0004] Therefore, there is an urgent need for an autonomous obstacle avoidance and path planning method for UAVs that integrates multimodal perception, intelligent decision-making, and collaborative control, in order to improve adaptability, safety, and mission efficiency in complex environments. Summary of the Invention

[0005] In view of this, it is necessary to provide a method and system for autonomous obstacle avoidance and dynamic path planning of UAVs based on multimodal perception and hybrid intelligent decision-making. Through multimodal data fusion and intelligent algorithm collaboration, the UAV can achieve autonomous, safe and efficient flight in complex environments, which has significant technical advantages and application value.

[0006] In a first aspect, embodiments of this application provide a method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles (UAVs) based on multimodal perception and hybrid intelligent decision-making, the method comprising:

[0007] S1. Multimodal environmental perception: Real-time acquisition of environmental parameters such as wind speed W, illuminance L, temperature and humidity H, and image quality index Q through airborne weather station, IMU and visual sensor. The image quality index includes sharpness, contrast and signal-to-noise ratio.

[0008] S2. Establish a flight parameter-image quality coupling model:

[0009] A nonlinear mapping from environmental factors E to image quality Q is constructed using Support Vector Regression (SVR). This nonlinear mapping accurately describes the complex relationship between environmental factors and image quality, enabling the UAV system to predict image quality based on different environmental factors. The nonlinear mapping is expressed as:

[0010] , ,

[0011] in, This represents the nonlinear mapping function constructed by Support Vector Regression (SVR). Represents a set of environmental factors;

[0012] Solving multi-objective optimization problems using Particle Swarm Optimization (PSO):

[0013] ,

[0014] in For speed, For height, This indicates the image quality score at the current speed and altitude. Let be a function related to speed and height, representing the obstacle recognition accuracy. The task completion time is related to path length and speed. These are weighting factors, representing the priorities of image quality, recognition accuracy, and task time, respectively.

[0015] S3, Dynamic Strategy Optimization:

[0016] Constructing the state space of Q-learning Action space ;

[0017] Design a reward function:

[0018] ,

[0019] in, express Instant rewards for each moment This indicates the current image quality score. Indicates the distance between the drone and the nearest obstacle. This indicates the preset safe distance threshold. Indicates the current energy consumption. , , These are the weighting coefficients for image quality, obstacle avoidance safety, and energy consumption, respectively, representing the priorities of image quality, obstacle avoidance safety, and energy consumption.

[0020] S4, Intelligent Obstacle Handling:

[0021] ORB-SLAM3 is used to build 3D obstacle models in real time and calculate hazard factors.

[0022] ,

[0023] in, Indicates the dynamic hazard factor of the obstacle. The speed of the obstacle's movement Indicates the angle between the direction of the obstacle's movement and the drone's heading. This represents the minimum predicted distance between the drone and the obstacle.

[0024] An improved APF-RRT* algorithm is used to generate obstacle avoidance trajectories. The potential field function includes:

[0025] ,

[0026] in, Let be the total potential field function. The potential field attracted to the target point is positively correlated with the distance to the target. This represents the potential field for image quality, which is negatively correlated with imaging performance. For a dynamic obstacle repulsive potential field, and Positive correlation These are adaptive weighting coefficients;

[0027] S5, Online Collaborative Control:

[0028] Real-time optimization of control parameters using the BFGS algorithm ,satisfy:

[0029] ,

[0030] in, Represents the control vector. Indicates the heading angle. This indicates the actual imaging quality index under the current control parameters. For ideal imaging quality indicators, This represents the reference control value, derived from global planning. This represents the regularization coefficient, which balances tracking accuracy and control smoothness.

[0031] Optionally, in one implementation of the first aspect of the present invention, in step S1, the collected environmental parameters and image quality index Q undergo dimensional preprocessing, specifically including:

[0032] Normalization: The Min-Max normalization method is used to map each parameter to the [0,1] interval, as shown in the following formula:

[0033] ,

[0034] in, These are the original parameters. These are the maximum and minimum values ​​of the parameter in the historical dataset, respectively;

[0035] Illuminance dynamic correction: If illuminance If there is a diurnal periodic variation, a sliding window mean method is used for dynamic benchmark correction, and the formula is:

[0036] ,

[0037] in, The average illuminance of the most recent N sampling windows;

[0038] Image quality fusion: The three sub-indicators of sharpness, contrast, and signal-to-noise ratio are weighted and summed to obtain a comprehensive image quality score. :

[0039] ,

[0040] Among them, the weighting coefficient Principal component analysis was used to determine the balance of contributions of each sub-indicator.

[0041] Optionally, in one implementation of the first aspect of the present invention, the step of solving the multi-objective optimization problem through particle swarm optimization (PSO) includes:

[0042] Step 1, Initialize the particle swarm: Initialize the position and velocity of the particle swarm, randomly generate the position and velocity of each particle, and the position of each particle... This represents a set of flight parameter combinations, where To optimize the dimensionality of variables, the population size is set. Maximum number of iterations ;

[0043] Step 2, define and normalize the multi-objective fitness function: The fitness function consists of multiple objective functions, and the formula for the fitness function is:

[0044] ,

[0045] in, Image quality is scored, and the output is predicted based on the SVR model; The accuracy of obstacle recognition is negatively correlated with flight altitude and speed. This indicates the task completion time, which is positively correlated with the path length. This is an energy consumption indicator, related to acceleration and wind speed;

[0046] Step 3, Iteratively update particle state: The particle's position and velocity are updated in each generation based on the individual's optimal position. and global optimal position Update speed and location The formula is as follows:

[0047] ,

[0048] ,

[0049] in, Inertial weights control the degree to which a particle depends on its current velocity; These are learning factors that control the degree to which a particle depends on its own experience and the experience of its peers, respectively. To generate random numbers, randomness is introduced to enhance the algorithm's exploratory capabilities;

[0050] Step 4, Non-dominated sorting and optimal solution selection:

[0051] Based on the Pareto dominance relation, the particles are sorted by fitness to generate a non-dominated solution set;

[0052] The Pareto optimal solution with uniform distribution is selected by crowding degree calculation;

[0053] Step 5, Algorithm Termination and Result Output:

[0054] When the maximum number of iterations is reached When the time is reached, the algorithm terminates, and at this point, the globally optimal position is found. This is the final optimal solution. In multi-objective optimization problems, the final optimal solution is usually a set of Pareto optimal solutions, i.e., the Pareto front.

[0055] Optionally, in one implementation of the first aspect of the present invention, the non-dominated sorting and optimal solution selection include:

[0056] Solving using Pareto dominance relations includes:

[0057] untie Dominate If and only if

[0058] ,and ,

[0059] Calculate the dominance relationships of all solutions to generate the first-level non-dominated front; remove the sorted solutions and iteratively generate the next-level front;

[0060] For solutions to the same frontier, sort them by distance in each objective space, and retain solutions with uniform distribution:

[0061] ,

[0062] in, Indicates the degree of crowding. This indicates that the individual solutions for the current congestion level to be calculated belong to the same non-dominated frontier layer and the same Pareto level. Indicates the number of objective functions. Solution In the The value of the objective function untie Adjacent solutions in the target space In the current non-dominated frontier, the first... The maximum and minimum values ​​of each objective function.

[0063] Optionally, in one implementation of the first aspect of the present invention, the step of constructing a three-dimensional obstacle model in real time using ORB-SLAM3 includes:

[0064] The system acquires environmental image sequences through a visual sensor and simultaneously collects motion data through an inertial measurement unit (IMU).

[0065] ORB feature points are extracted from the image to generate keyframes with high discriminative power. Visual-inertial joint optimization is performed based on maximum a posteriori probability (MAP) to build a 3D sparse point cloud map containing obstacle features.

[0066] Achieve multi-level data association: Short-term association: Match obstacle feature points within the current field of view; Medium-term association: Optimize local obstacle models using shared keyframes; Long-term association: Match obstacle information from historical maps using an improved location recognition algorithm.

[0067] When the system loses tracking, it automatically creates a new map and seamlessly merges multiple maps through non-linear optimization when it detects repeated obstacle areas.

[0068] The Bundle Adjustment algorithm is used to optimize the 3D model of obstacles in real time, and output the spatial coordinates of obstacles with centimeter-level accuracy.

[0069] Optionally, in one implementation of the first aspect of the present invention, the short-term association includes: extracting ORB feature points of the current frame image; predicting the feature point search region using an inertial measurement unit (IMU) pre-integration motion model; matching obstacle feature points within the current field of view using Hamming distance; and eliminating mismatches based on epipolar geometric constraints and depth consistency checks.

[0070] Optionally, in one implementation of the first aspect of the present invention, the intermediate association includes: constructing a weighted co-view relationship matrix, dynamically selecting co-view keyframes to form a local map subset; and optimizing the pose of the co-view keyframes through local BA. and the three-dimensional point coordinates of local obstacles The objective function is optimized as follows:

[0071] ,

[0072] in, Here, π is the Huber robust kernel function, and π is the projection function. To observe feature points, These are the coordinates of a three-dimensional point on a local obstacle.

[0073] Optionally, in one implementation of the first aspect of the present invention, the long-term association includes: matching obstacle information in historical maps using a three-level retrieval architecture; performing initial visual similarity screening based on the DBOW bag-of-words model; calculating a comprehensive matching score by fusing IMU motion consistency; verifying cross-map obstacle correspondence through similarity transformation; and further includes a multi-map fusion module: when a matching obstacle region is detected between the historical map and the current map, constructing a cross-map factor map for optimization; performing a bidirectional projection consistency check and setting a projection error threshold; and progressively fusing obstacle models to ensure global consistency. Figure 1 To the point of being responsive.

[0074] Secondly, embodiments of this application provide an autonomous obstacle avoidance and dynamic path planning system for unmanned aerial vehicles (UAVs) based on multimodal perception and hybrid intelligent decision-making, applied to the autonomous obstacle avoidance and dynamic path planning method for UAVs based on multimodal perception and hybrid intelligent decision-making as described in the first aspect. The system includes:

[0075] Multimodal environmental perception module: Real-time acquisition of environmental parameters such as wind speed W, illuminance L, temperature and humidity H, and image quality index Q through airborne weather station, IMU and visual sensor. The image quality index includes sharpness, contrast and signal-to-noise ratio.

[0076] Flight parameters-image quality coupling modeling module:

[0077] A nonlinear mapping from environmental factors E to image quality Q is constructed using Support Vector Regression (SVR). This nonlinear mapping accurately describes the complex relationship between environmental factors and image quality, enabling the UAV system to predict image quality based on different environmental factors. The nonlinear mapping is expressed as:

[0078] , ,

[0079] in, This represents the nonlinear mapping function constructed by Support Vector Regression (SVR). Represents a set of environmental factors;

[0080] Solving multi-objective optimization problems using Particle Swarm Optimization (PSO):

[0081] ,

[0082] in For speed, For height, This indicates the image quality score at the current speed and altitude. Let be a function related to speed and height, representing the obstacle recognition accuracy. The task completion time is related to path length and speed. These are weighting factors, representing the priorities of image quality, recognition accuracy, and task time, respectively.

[0083] Dynamic strategy optimization module:

[0084] Constructing the state space of Q-learning Action space ;

[0085] Design a reward function:

[0086] ,

[0087] in, express Instant rewards for each moment This indicates the current image quality score. Indicates the distance between the drone and the nearest obstacle. This indicates the preset safe distance threshold. Indicates the current energy consumption. , , These are the weighting coefficients for image quality, obstacle avoidance safety, and energy consumption, respectively, representing the priorities of image quality, obstacle avoidance safety, and energy consumption.

[0088] Intelligent obstacle processing module:

[0089] ORB-SLAM3 is used to build 3D obstacle models in real time and calculate hazard factors.

[0090] ,

[0091] in, Indicates the dynamic hazard factor of the obstacle. The speed of the obstacle's movement Indicates the angle between the direction of the obstacle's movement and the drone's heading. This represents the minimum predicted distance between the drone and the obstacle.

[0092] An improved APF-RRT* algorithm is used to generate obstacle avoidance trajectories. The potential field function includes:

[0093] ,

[0094] in, Let be the total potential field function. The potential field attracted to the target point is positively correlated with the distance to the target. This represents the potential field for image quality, which is negatively correlated with imaging performance. For a dynamic obstacle repulsive potential field, and Positive correlation These are adaptive weighting coefficients;

[0095] Online collaborative control module:

[0096] Real-time optimization of control parameters using the BFGS algorithm ,satisfy:

[0097] ,

[0098] in, Represents the control vector. Indicates the heading angle. This indicates the actual imaging quality index under the current control parameters. For ideal imaging quality indicators, This represents the reference control value, derived from global planning. This represents the regularization coefficient, which balances tracking accuracy and control smoothness.

[0099] Thirdly, embodiments of this application provide an electronic device, including:

[0100] processor;

[0101] Memory used to store processor-executable instructions;

[0102] The processor is configured to implement the UAV autonomous obstacle avoidance and dynamic path planning method based on multimodal perception and hybrid intelligent decision-making as described in the first aspect when executing the instructions.

[0103] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to execute the UAV autonomous obstacle avoidance and dynamic path planning method based on multimodal perception and hybrid intelligent decision-making as described in the first aspect.

[0104] This invention proposes a method and system for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles (UAVs) based on multimodal perception and hybrid intelligent decision-making. Environmental parameters such as wind speed, illuminance, temperature, and humidity, as well as image quality indicators, are collected in real time through an airborne weather station, IMU, and visual sensors. A flight parameter-image quality coupling model is established, and multi-objective optimization is achieved using Support Vector Regression (SVR) and Particle Swarm Optimization (PSO). A dynamic strategy optimization module is designed based on Q-learning, and a reward function is designed combining image quality, obstacle avoidance safety, and energy consumption. A 3D obstacle model is constructed in real time using ORB-SLAM3, dynamic hazard coefficients are calculated, and an improved APF-RRT* algorithm is used to generate obstacle avoidance trajectories. An online collaborative control module uses the BFGS algorithm to optimize control variables, ensuring flight stability and mission efficiency. This invention achieves high-precision obstacle avoidance and path planning for UAVs in complex environments, exhibiting strong robustness and wide adaptability, and is suitable for practical applications such as inspection and surveying. Through multimodal data fusion and intelligent algorithm collaboration, this invention enables autonomous, safe, and efficient flight of UAVs in complex environments, demonstrating significant technical advantages and application value.

[0105] Beneficial effects:

[0106] (1) Multimodal environmental perception and intelligent decision-making: By integrating meteorological data, IMU motion information and visual sensor data, comprehensive perception of environmental factors is achieved, enabling UAVs to dynamically adjust their flight strategies according to real-time environmental changes, significantly improving their adaptability in complex environments.

[0107] (2) Balancing image quality optimization and flight efficiency: Support vector regression (SVR) and particle swarm optimization (PSO) algorithms were used to establish a coupling model between flight parameters and image quality. While ensuring image acquisition quality, flight speed and altitude were optimized, thus balancing mission efficiency and data quality.

[0108] (3) Dynamic obstacle avoidance and path planning: A high-precision 3D obstacle model is constructed in real time using ORB-SLAM3, and combined with the improved APF-RRT* algorithm, effective avoidance of both static and dynamic obstacles is achieved. The introduction of dynamic hazard coefficients makes the obstacle avoidance strategy more intelligent and significantly improves flight safety.

[0109] (4) Multi-objective optimization and energy consumption control: Based on the Q-learning reward function design, the image quality, obstacle avoidance safety and energy consumption factors are comprehensively considered, so that the UAV can optimize energy consumption and extend the endurance while completing the mission.

[0110] (5) Real-time performance and robustness: The BFGS algorithm is used for online collaborative control, which ensures the real-time performance and stability of flight control and maintains a smooth flight trajectory even in complex and changeable environments.

[0111] (6) High system integration: It organically integrates functional modules such as environmental perception, path planning, obstacle avoidance decision and flight control to form a complete autonomous flight solution with broad application prospects, such as inspection, surveying and mapping, disaster relief and other fields. Attached Figure Description

[0112] Figure 1 This is a schematic diagram of a method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles based on multimodal perception and hybrid intelligent decision-making, provided in an embodiment of this application.

[0113] Figure 2 This is a flowchart illustrating a method for solving multi-objective optimization problems using particle swarm optimization (PSO) according to an embodiment of this application.

[0114] Figure 3 This is a system architecture diagram of an unmanned aerial vehicle (UAV) autonomous obstacle avoidance and dynamic path planning based on multimodal perception and hybrid intelligent decision-making, provided as an embodiment of this application.

[0115] Figure 4 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0116] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0117] It should be noted that, in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0118] It should be noted that in the embodiments of this application, the terms "first," "second," etc., are used only for descriptive purposes and should not be construed as indicating or implying relative importance, nor as indicating or implying order. Features specified as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0119] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0120] Example 1

[0121] Figure 1 This is a schematic flowchart illustrating a method for autonomous obstacle avoidance and dynamic path planning of an unmanned aerial vehicle (UAV) based on multimodal perception and hybrid intelligent decision-making, provided as an embodiment of this application. Figure 1 As shown, a method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles (UAVs) based on multimodal perception and hybrid intelligent decision-making includes:

[0122] S1, Multimodal environment perception.

[0123] The system collects environmental parameters such as wind speed (W), illuminance (L), temperature and humidity (H), and image quality index (Q) in real time through an airborne weather station, IMU, and visual sensors. The image quality index includes sharpness, contrast, and signal-to-noise ratio.

[0124] Specifically, environmental parameters such as wind speed (W), illuminance (L), temperature and humidity (H), and image quality metrics (Q) are collected in real time using an airborne weather station, an IMU (Integrated Measurement Unit), and visual sensors. These image quality metrics include sharpness, contrast, and signal-to-noise ratio. Multimodal sensing technology allows for the real-time acquisition of environmental parameters and comprehensive analysis in conjunction with image quality metrics. For example, the airborne weather station can collect environmental parameters such as wind speed, illuminance, temperature, and humidity, which are crucial for understanding the current environmental conditions. Furthermore, the IMU provides information such as acceleration and angular velocity to assist in positioning and attitude estimation. Visual sensors are used to collect image data, the quality of which directly affects subsequent environmental understanding and decision-making. Multimodal sensing technology allows for the fusion of data from different sensors, resulting in more comprehensive and accurate environmental information.

[0125] In step S1, the collected environmental parameters and image quality index Q undergo dimensional preprocessing, specifically including: normalization processing: using the Min-Max normalization method, each parameter is mapped to the [0,1] interval. This method improves the comparability of data by eliminating the dimensional influence between data indicators. The formula is as follows:

[0126] ,

[0127] in, These are the original parameters. These are the maximum and minimum values ​​of the parameter in the historical dataset, respectively;

[0128] Illuminance dynamic correction: If illuminance If there is a diurnal periodic variation, a sliding window averaging method is used for dynamic benchmark correction. This method calculates the average illuminance of the most recent N sampling windows to eliminate the impact of periodic illuminance variations on image quality. The formula is:

[0129] ,

[0130] in, This represents the average illuminance across the most recent N sampling windows.

[0131] Image quality fusion: The three sub-indicators of sharpness, contrast, and signal-to-noise ratio are weighted and summed to obtain a comprehensive image quality score. : Among them, the weighting coefficient Principal component analysis was used to determine the balance of contributions of each sub-indicator.

[0132] The above preprocessing steps can effectively improve the comparability and consistency of environmental parameters and image quality indicators, providing reliable data support for the subsequent establishment of multimodal environment perception and flight parameter-image quality coupling models.

[0133] S2. Establish a flight parameter-image quality coupling model.

[0134] A nonlinear mapping from environmental factors E to image quality Q is constructed using Support Vector Regression (SVR). This nonlinear mapping accurately describes the complex relationship between environmental factors and image quality, enabling the UAV system to predict image quality based on different environmental factors. The nonlinear mapping is expressed as:

[0135] , ,

[0136] in, This represents the nonlinear mapping function constructed by Support Vector Regression (SVR). It represents a set of environmental factors.

[0137] Support vector regression (SVR) is used to construct a nonlinear mapping from environmental factors E to image quality Q. This nonlinear mapping can accurately describe the complex relationship between environmental factors and image quality, enabling UAV systems to predict image quality based on different environmental factors.

[0138] Solving multi-objective optimization problems using Particle Swarm Optimization (PSO):

[0139] ,

[0140] in For speed, For height, This indicates the image quality score at the current speed and altitude. Let be a function related to speed and height, representing the obstacle recognition accuracy. The task completion time is related to path length and speed. The weighting factors represent the priorities of image quality, recognition accuracy, and task time, respectively. A multi-objective optimization problem is solved using Particle Swarm Optimization (PSO), where velocity and altitude represent the image quality scores at the current velocity and altitude, respectively. Obstacle recognition accuracy, task completion time, path length, and velocity are related to task completion time, and the weighting factors represent the priorities of image quality, recognition accuracy, and task time, respectively. Through multi-objective optimization, an optimal solution that balances image quality, obstacle recognition accuracy, and task completion time can be found, thereby improving the overall performance of the UAV system.

[0141] A nonlinear mapping is constructed to clarify the relationship between environmental factors and image quality. Support vector regression (SVR) is used to construct a nonlinear mapping from environmental factor E to image quality Q. During UAV flight, environmental factors are complex and diverse, such as wind speed, illumination, temperature, and humidity, all of which affect image quality. This nonlinear mapping can more accurately describe the complex relationship between environmental factors and image quality, allowing the UAV system to predict image quality based on different environmental factors.

[0142] Changes in environmental factors directly affect image quality metrics such as sharpness, contrast, and signal-to-noise ratio. By comprehensively considering multiple factors, establishing this mapping relationship helps UAVs assess and predict image quality under different environments, providing a basis for subsequent decision-making.

[0143] Solving a multi-objective optimization problem to optimize flight parameters and improve mission performance. Particle Swarm Optimization (PSO) is used to solve the multi-objective optimization problem, considering multiple factors such as speed, altitude, obstacle recognition accuracy, and mission completion time. Speed ​​and altitude are crucial parameters for UAV flight, affecting not only flight efficiency but also image quality.

[0144] By introducing weighting factors, which represent the priorities of image quality, recognition accuracy, and task time, respectively, different tasks may have different emphases on these indicators during actual flight. For example, some tasks may prioritize image quality, while others may focus more on task completion time. By adjusting the weighting factors, the UAV can find the optimal combination of flight parameters under different task requirements, thereby improving the overall mission performance.

[0145] The establishment of the flight parameter-image quality coupling model provides an important foundation for subsequent dynamic policy optimization. In dynamic policy optimization, factors such as image quality need to be considered when designing the reward function to guide the UAV to make optimal decisions.

[0146] In obstacle intelligent processing, accurate image quality is crucial for real-time construction of obstacle 3D models and calculation of hazard factors. Improving image quality by optimizing flight parameters can provide more reliable data for obstacle intelligent processing, thereby enhancing the obstacle avoidance capabilities and safety of drones.

[0147] Figure 2 This is a flowchart illustrating a method for solving multi-objective optimization problems using Particle Swarm Optimization (PSO) according to an embodiment of this application. Specifically, as shown... Figure 2 As shown, the method of solving the multi-objective optimization problem using particle swarm optimization (PSO) includes:

[0148] Step 1, Initialize the particle swarm: Initialize the position and velocity of the particle swarm, randomly generate the position and velocity of each particle, and the position of each particle... This represents a set of flight parameter combinations, where To optimize the dimensionality of variables, the population size is set. Maximum number of iterations .

[0149] Step 2, define and normalize the multi-objective fitness function: The fitness function consists of multiple objective functions, and the formula for the fitness function is:

[0150] ,

[0151] in, Image quality is scored, and the output is predicted based on the SVR model; The accuracy of obstacle recognition is negatively correlated with flight altitude and speed. This indicates the task completion time, which is positively correlated with the path length. It is an energy consumption indicator, related to acceleration and wind speed.

[0152] Step 3, Iteratively update particle state: The particle's position and velocity are updated in each generation based on the individual's optimal position. and global optimal position Update speed and location The formula is as follows:

[0153] ,

[0154] ,

[0155] in, Inertial weights control the degree to which a particle depends on its current velocity; These are learning factors that control the degree to which a particle depends on its own experience and the experience of its peers, respectively. To introduce randomness, we use random numbers to enhance the algorithm's exploratory capabilities.

[0156] Step 4, Non-dominated sorting and optimal solution selection: Sort the particles by fitness according to Pareto dominance to generate a set of non-dominated solutions; select uniformly distributed Pareto optimal solutions by crowding calculation.

[0157] Step 5, Algorithm Termination and Result Output: When the maximum number of iterations is reached... When the time is reached, the algorithm terminates, and at this point, the globally optimal position is found. This is the final optimal solution. In multi-objective optimization problems, the final optimal solution is usually a set of Pareto optimal solutions, i.e., the Pareto front.

[0158] Specifically, the non-dominated sorting and optimal solution selection include: setting Pareto dominance relations for solving, including: solution Dominate If and only if

[0159] ,and ,

[0160] Calculate the dominance relationships of all solutions to generate the first layer of non-dominated fronts; remove the sorted solutions and iteratively generate the next layer of fronts; for solutions on the same front, sort them according to their distance from each objective space, retaining solutions with uniform distribution: ,in, Indicates the degree of crowding. This indicates that the individual solutions for the current congestion level to be calculated belong to the same non-dominated frontier layer and the same Pareto level. Indicates the number of objective functions. Solution In the The value of the objective function untie Adjacent solutions in the target space In the current non-dominated frontier, the first... The maximum and minimum values ​​of each objective function.

[0161] S3, Dynamic Strategy Optimization:

[0162] Constructing the state space of Q-learning Action space ;

[0163] Design a reward function:

[0164] ,

[0165] in, express Instant rewards for each moment This indicates the current image quality score. Indicates the distance between the drone and the nearest obstacle. This indicates the preset safe distance threshold. Indicates the current energy consumption. , , These are the weighting coefficients for image quality, obstacle avoidance safety, and energy consumption, respectively, representing the priorities of image quality, obstacle avoidance safety, and energy consumption.

[0166] A Q-learning state space and action space are constructed, and a reward function is designed where the immediate reward is related to the current image quality score, the distance between the UAV and the nearest obstacle, a preset safe distance threshold, and the current power consumption. Weight coefficients represent the priorities of image quality, obstacle avoidance safety, and power consumption, respectively. Through the Q-learning algorithm, the UAV can learn strategies for taking optimal actions in different states, thereby achieving efficient obstacle avoidance and task execution. By continuously updating the state-action value function, the Q-learning algorithm enables the UAV to make optimal decisions in complex environments, improving obstacle avoidance efficiency and task completion rate.

[0167] S4, Intelligent obstacle handling.

[0168] ORB-SLAM3 is used to build 3D obstacle models in real time and calculate hazard factors.

[0169] ,

[0170] in, Indicates the dynamic hazard factor of the obstacle. The speed of the obstacle's movement Indicates the angle between the direction of the obstacle's movement and the drone's heading. This represents the minimum predicted distance between the drone and the obstacle.

[0171] An improved APF-RRT* algorithm is used to generate obstacle avoidance trajectories. The potential field function includes:

[0172] ,

[0173] in, Let be the total potential field function. The potential field attracted to the target point is positively correlated with the distance to the target. This represents the potential field for image quality, which is negatively correlated with imaging performance. For a dynamic obstacle repulsive potential field, and Positive correlation These are adaptive weighting coefficients.

[0174] A 3D obstacle model is constructed in real time using ORB-SLAM3, and the hazard factor is calculated. The dynamic hazard factor of the obstacle is related to the obstacle's movement speed, the angle between the obstacle's movement direction and the UAV's heading, and the minimum predicted distance between the UAV and the obstacle. An improved APF-RRT is employed. The algorithm generates obstacle avoidance trajectories, and the potential field function includes the attractive potential field of the target point and the image quality. Potential field, dynamic obstacle repulsive potential field, and adaptive weighting coefficients. This is achieved through an improved APF-RRT. The algorithm enables UAVs to generate an optimal path to avoid obstacles while considering image quality and obstacle avoidance safety. The improved APF-RRT* algorithm dynamically adjusts the potential field function, allowing UAVs to avoid obstacles in complex environments in real time, thus improving flight safety and mission completion rate.

[0175] Specifically, the real-time construction of a 3D obstacle model using ORB-SLAM3 includes: acquiring environmental image sequences via a visual sensor and simultaneously collecting motion data via an inertial measurement unit (IMU); extracting ORB feature points from the images, generating highly discriminative keyframes, and performing joint visual-inertial optimization based on maximum a posteriori probability (MAP) to establish a 3D sparse point cloud map containing obstacle features; implementing multi-level data association: short-term association: matching obstacle feature points within the current field of view; medium-term association: optimizing local obstacle models using shared keyframes; long-term association: matching obstacle information from historical maps using an improved location recognition algorithm; automatically creating a new map when the system loses tracking, and seamlessly fusing multiple maps through nonlinear optimization when duplicate obstacle areas are detected; and performing real-time optimization of the 3D obstacle model based on the Bundle Adjustment algorithm to output obstacle spatial coordinates with centimeter-level accuracy.

[0176] The short-term association includes: extracting ORB feature points from the current frame image; predicting the feature point search region using the inertial measurement unit (IMU) pre-integration motion model; matching obstacle feature points within the current field of view using Hamming distance; and eliminating mismatches based on epipolar geometric constraints and depth consistency checks.

[0177] The intermediate association includes: constructing a weighted co-view relationship matrix, dynamically selecting co-view keyframes to form a local map subset; and optimizing the pose of the co-view keyframes through local BA. and the three-dimensional point coordinates of local obstacles The objective function is optimized as follows:

[0178] ,

[0179] in, Here, π is the Huber robust kernel function, and π is the projection function. To observe feature points, These are the coordinates of a three-dimensional point on a local obstacle.

[0180] The long-term association includes: using a three-level retrieval architecture to match obstacle information in historical maps; performing initial visual similarity screening based on the DBOW bag-of-words model; calculating a comprehensive matching score by fusing IMU motion consistency; and verifying the correspondence between obstacles across maps through similarity transformation.

[0181] It also includes a multi-map fusion module: when a matching obstacle region is detected between the historical map and the current map, a cross-map factor map is constructed for optimization; a bidirectional projection consistency check is performed, and a projection error threshold is set; obstacle models are progressively fused to ensure global consistency. Figure 1 To the point of being responsive.

[0182] S5, Online Collaborative Control:

[0183] Real-time optimization of control parameters using the BFGS algorithm ,satisfy:

[0184] ,

[0185] in, Represents the control vector. Indicates the heading angle. This indicates the actual imaging quality index under the current control parameters. For ideal imaging quality indicators, This represents the reference control value, derived from global planning. This represents the regularization coefficient, which balances tracking accuracy and control smoothness.

[0186] The BFGS algorithm optimizes control variables in real time, satisfying the relationships between control vector, control velocity, control altitude, heading angle, actual imaging quality index under the current control variable, ideal imaging quality index, reference control variable, and regularization coefficient. Through the BFGS algorithm, the UAV can adjust control variables in real time to meet the requirements of the flight mission while maintaining stable imaging quality. The BFGS algorithm, through iterative optimization, enables the UAV to adjust control variables in real time in complex environments, improving flight stability and imaging quality.

[0187] Through multimodal environment perception, flight parameter-image quality coupling model, dynamic strategy optimization, intelligent obstacle handling, and online collaborative control, unmanned aerial vehicle (UAV) systems can achieve efficient obstacle avoidance and mission execution in complex environments.

[0188] Example 2

[0189] like Figure 3As shown, this application provides an architecture diagram of an autonomous obstacle avoidance and dynamic path planning system for unmanned aerial vehicles (UAVs) based on multimodal perception and hybrid intelligent decision-making. This system is applied to the autonomous obstacle avoidance and dynamic path planning system for UAVs based on multimodal perception and hybrid intelligent decision-making as described in Embodiment 1. It includes a multimodal environment perception module 11, a flight parameter-image quality coupling modeling module 12, a dynamic strategy optimization module 13, an obstacle intelligent processing module 14, and an online collaborative control module 15.

[0190] The multimodal environment perception module 11 is used to collect environmental parameters such as wind speed W, illuminance L, temperature and humidity H, and image quality index Q in real time through an airborne weather station, IMU, and visual sensors. The image quality index includes sharpness, contrast, and signal-to-noise ratio.

[0191] The flight parameter-image quality coupling modeling module 12 is used to construct a nonlinear mapping from environmental factors E to image quality Q using support vector regression (SVR). This nonlinear mapping accurately describes the complex relationship between environmental factors and image quality, enabling the UAV system to predict image quality based on different environmental factors. The nonlinear mapping is expressed as follows:

[0192] , ,

[0193] in, This represents the nonlinear mapping function constructed by Support Vector Regression (SVR). Represents a set of environmental factors;

[0194] Solving multi-objective optimization problems using Particle Swarm Optimization (PSO):

[0195] ,

[0196] in For speed, For height, This indicates the image quality score at the current speed and altitude. Let be a function related to speed and height, representing the obstacle recognition accuracy. The task completion time is related to path length and speed. These are weighting factors, representing the priorities of image quality, recognition accuracy, and task time, respectively.

[0197] Dynamic policy optimization module 13 is used to construct the state space of Q-learning. Action space ;

[0198] Design a reward function:

[0199] ,

[0200] in, express Instant rewards for each moment This indicates the current image quality score. Indicates the distance between the drone and the nearest obstacle. This indicates the preset safe distance threshold. Indicates the current energy consumption. , , These are the weighting coefficients for image quality, obstacle avoidance safety, and energy consumption, respectively, representing the priorities of image quality, obstacle avoidance safety, and energy consumption.

[0201] The obstacle intelligent processing module 14 is used to construct a 3D model of obstacles in real time using ORB-SLAM3 and calculate the hazard factor.

[0202] ,

[0203] in, Indicates the dynamic hazard factor of the obstacle. The speed of the obstacle's movement Indicates the angle between the direction of the obstacle's movement and the drone's heading. This represents the minimum predicted distance between the drone and the obstacle.

[0204] An improved APF-RRT* algorithm is used to generate obstacle avoidance trajectories. The potential field function includes:

[0205] ,

[0206] in, Let be the total potential field function. The potential field attracted to the target point is positively correlated with the distance to the target. This represents the potential field for image quality, which is negatively correlated with imaging performance. For a dynamic obstacle repulsive potential field, and Positive correlation These are adaptive weighting coefficients;

[0207] The online collaborative control module 15 is used to optimize the control input in real time using the BFGS algorithm. ,satisfy:

[0208] ,

[0209] in, Represents the control vector. Indicates speed control. Indicates control height. Indicates the heading angle. This indicates the actual imaging quality index under the current control parameters. For ideal imaging quality indicators, This represents the reference control value, derived from global planning. This represents the regularization coefficient, which balances tracking accuracy and control smoothness.

[0210] Figure 4 This is an electronic device provided in one embodiment of this application. For example... Figure 4 As shown, the electronic device includes at least the following components: processor 101 and memory 100, communication interface 103, and bus 102.

[0211] In this embodiment of the application, memory 100 is used to store executable instructions of processor 101, which, when configured to execute instructions, implements the method as described in the first aspect.

[0212] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.

[0213] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these devices is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0214] It should be noted that a portion of the electronic device described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0215] It should be noted that the term "computer" as used here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, "computer-readable recording media" refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard drives built into a computer.

[0216] Furthermore, a "computer-readable recording medium" can include: a medium that dynamically stores a program for a short period of time, such as a communication line used when transmitting a program via a network such as the Internet or a communication line such as a telephone line; or a medium that stores a program for a fixed period of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining with programs already recorded in the computer.

[0217] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (device group) composed of multiple devices. Each device constituting the device group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a device group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0218] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles (UAVs) based on multimodal perception and hybrid intelligent decision-making, characterized in that, The method includes: S1. Multimodal environmental perception: Real-time acquisition of environmental parameters such as wind speed W, illuminance L, temperature and humidity H, and image quality index Q through airborne weather station, IMU and visual sensor. The image quality index includes sharpness, contrast and signal-to-noise ratio. S2. Establish a flight parameter-image quality coupling model: A nonlinear mapping from environmental factors E to image quality Q is constructed using Support Vector Regression (SVR). This nonlinear mapping accurately describes the complex relationship between environmental factors and image quality, enabling the UAV system to predict image quality based on different environmental factors. The nonlinear mapping is expressed as: , ， in, This represents the nonlinear mapping function constructed by Support Vector Regression (SVR). Represents a set of environmental factors; Solving multi-objective optimization problems using Particle Swarm Optimization (PSO): ， in For speed, For height, This indicates the image quality score at the current speed and altitude. Let be a function related to speed and height, representing the obstacle recognition accuracy. The task completion time is related to path length and speed. These are weighting factors, representing the priorities of image quality, recognition accuracy, and task time, respectively. S3, Dynamic Strategy Optimization: Constructing the state space of Q-learning Action space ; Design a reward function: ， in, express Instant rewards for each moment This indicates the current image quality score. Indicates the distance between the drone and the nearest obstacle. This indicates the preset safe distance threshold. Indicates the current energy consumption. , , These are the weighting coefficients for image quality, obstacle avoidance safety, and energy consumption, respectively, representing the priorities of image quality, obstacle avoidance safety, and energy consumption. S4, Intelligent Obstacle Handling: ORB-SLAM3 is used to build 3D obstacle models in real time and calculate hazard factors. ， in, Indicates the dynamic hazard factor of the obstacle. The speed of the obstacle's movement Indicates the angle between the direction of the obstacle's movement and the drone's heading. This represents the minimum predicted distance between the drone and the obstacle. An improved APF-RRT* algorithm is used to generate obstacle avoidance trajectories. The potential field function includes: ， in, Let be the total potential field function. The potential field attracted to the target point is positively correlated with the distance to the target. This represents the potential field for image quality, which is negatively correlated with imaging performance. For a dynamic obstacle repulsive potential field, and Positive correlation These are adaptive weighting coefficients; S5, Online Collaborative Control: Real-time optimization of control parameters using the BFGS algorithm ,satisfy: ， in, Represents the control vector. Indicates the heading angle. This indicates the actual imaging quality index under the current control parameters. For ideal imaging quality indicators, This represents the reference control value, derived from global planning. This represents the regularization coefficient, which balances tracking accuracy and control smoothness.

2. The method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles based on multimodal perception and hybrid intelligent decision-making according to claim 1, characterized in that, In step S1, the collected environmental parameters and image quality index Q undergo dimensional preprocessing, specifically including: Normalization: The Min-Max normalization method is used to map each parameter to the [0,1] interval, as shown in the following formula: ， in, These are the original parameters. These are the maximum and minimum values ​​of the parameter in the historical dataset, respectively; Illuminance dynamic correction: If illuminance If there is a diurnal periodic variation, a sliding window mean method is used for dynamic benchmark correction, and the formula is: ， in, The average illuminance of the most recent N sampling windows; Image quality fusion: The three sub-indicators of sharpness, contrast, and signal-to-noise ratio are weighted and summed to obtain a comprehensive image quality score. : ， Among them, the weighting coefficient Principal component analysis was used to determine the balance of contributions of each sub-indicator.

3. The method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles based on multimodal perception and hybrid intelligent decision-making according to claim 2, characterized in that, The method of solving multi-objective optimization problems using particle swarm optimization (PSO) includes: Step 1, Initialize the particle swarm: Initialize the position and velocity of the particle swarm, randomly generate the position and velocity of each particle, and the position of each particle... This represents a set of flight parameter combinations, where To optimize the dimensionality of variables, the population size is set. Maximum number of iterations ; Step 2, define and normalize the multi-objective fitness function: The fitness function consists of multiple objective functions, and the formula for the fitness function is: ， in, Image quality is scored, and the output is predicted based on the SVR model; The accuracy of obstacle recognition is negatively correlated with flight altitude and speed. This indicates the task completion time, which is positively correlated with the path length. This is an energy consumption indicator, related to acceleration and wind speed; Step 3, Iteratively update particle state: The particle's position and velocity are updated in each generation based on the individual's optimal position. and global optimal position Update speed and location The formula is as follows: ， ， in, Inertial weights control the degree to which a particle depends on its current velocity; These are learning factors that control the degree to which a particle depends on its own experience and the experience of its peers, respectively. To generate random numbers, randomness is introduced to enhance the algorithm's exploratory capabilities; Step 4, Non-dominated sorting and optimal solution selection: Based on the Pareto dominance relation, the particles are sorted by fitness to generate a non-dominated solution set; The Pareto optimal solution with uniform distribution is selected by crowding degree calculation; Step 5, Algorithm Termination and Result Output: When the maximum number of iterations is reached When the time is reached, the algorithm terminates, and at this point, the globally optimal position is found. This is the final optimal solution. In multi-objective optimization problems, the final optimal solution is a set of Pareto optimal solutions, i.e., the Pareto front.

4. The method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles based on multimodal perception and hybrid intelligent decision-making according to claim 3, characterized in that, The non-dominated sorting and optimal solution selection include: Solving using Pareto dominance relations includes: untie Dominate If and only if ,and , Calculate the dominance relationships of all solutions to generate the first-level non-dominated front; remove the sorted solutions and iteratively generate the next-level front; For solutions to the same frontier, sort them by distance in each objective space, and retain solutions with uniform distribution: ， in, Indicates the degree of crowding. This indicates that the individual solutions for the current congestion level to be calculated belong to the same non-dominated frontier layer and the same Pareto level. Indicates the number of objective functions. Solution In the The value of the objective function untie Adjacent solutions in the target space In the current non-dominated frontier, the first... The maximum and minimum values ​​of each objective function.

5. The method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles based on multimodal perception and hybrid intelligent decision-making according to claim 1, characterized in that, The method of constructing a 3D obstacle model in real time using ORB-SLAM3 includes: The system acquires environmental image sequences through a visual sensor and simultaneously collects motion data through an inertial measurement unit (IMU). ORB feature points are extracted from the image to generate keyframes with high discriminative power. Visual-inertial joint optimization is performed based on maximum a posteriori probability (MAP) to build a 3D sparse point cloud map containing obstacle features. Achieve multi-level data association: Short-term association: Match obstacle feature points within the current field of view; Medium-term association: Optimize local obstacle models using shared keyframes; Long-term association: Match obstacle information from historical maps using an improved location recognition algorithm. When the system loses tracking, it automatically creates a new map and seamlessly merges multiple maps through non-linear optimization when it detects repeated obstacle areas. The Bundle Adjustment algorithm is used to optimize the 3D model of obstacles in real time, and output the spatial coordinates of obstacles with centimeter-level accuracy.

6. The method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles based on multimodal perception and hybrid intelligent decision-making according to claim 5, characterized in that: The short-term association includes: extracting ORB feature points from the current frame image; predicting the feature point search region using the inertial measurement unit (IMU) pre-integration motion model; matching obstacle feature points within the current field of view using Hamming distance; and eliminating mismatches based on epipolar geometric constraints and depth consistency checks.

7. The method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles based on multimodal perception and hybrid intelligent decision-making according to claim 5, characterized in that: The intermediate association includes: constructing a weighted co-view relationship matrix, dynamically selecting co-view keyframes to form a local map subset; and optimizing the pose of the co-view keyframes through local BA. and the three-dimensional point coordinates of local obstacles The objective function is optimized as follows: ， in, Here, π is the Huber robust kernel function, and π is the projection function. To observe feature points, These are the coordinates of a three-dimensional point on a local obstacle.

8. The method for autonomous obstacle avoidance and dynamic path planning of unmanned aerial vehicles based on multimodal perception and hybrid intelligent decision-making according to claim 5, characterized in that: The long-term association includes: using a three-level retrieval architecture to match obstacle information in historical maps; performing initial visual similarity screening based on the DBOW bag-of-words model; calculating a comprehensive matching score by fusing IMU motion consistency; verifying the correspondence between obstacles across maps through similarity transformation; and also includes a multi-map fusion module: when a matching obstacle region is detected between the historical map and the current map, a cross-map factor map is constructed for optimization; performing a bidirectional projection consistency check and setting a projection error threshold; and progressively fusing obstacle models to ensure global map consistency.

9. A UAV autonomous obstacle avoidance and dynamic path planning system based on multimodal perception and hybrid intelligent decision-making, applied to the UAV autonomous obstacle avoidance and dynamic path planning method based on multimodal perception and hybrid intelligent decision-making as described in any one of claims 1 to 8, characterized in that, The system includes: Multimodal environmental perception module: Real-time acquisition of environmental parameters such as wind speed W, illuminance L, temperature and humidity H, and image quality index Q through airborne weather station, IMU and visual sensor. The image quality index includes sharpness, contrast and signal-to-noise ratio. Flight parameters-image quality coupling modeling module: A nonlinear mapping from environmental factors E to image quality Q is constructed using Support Vector Regression (SVR). This nonlinear mapping accurately describes the complex relationship between environmental factors and image quality, enabling the UAV system to predict image quality based on different environmental factors. The nonlinear mapping is expressed as: , ， in, This represents the nonlinear mapping function constructed by Support Vector Regression (SVR). Represents a set of environmental factors; Solving multi-objective optimization problems using Particle Swarm Optimization (PSO): ， in For speed, For height, This indicates the image quality score at the current speed and altitude. Let be a function related to speed and height, representing the obstacle recognition accuracy. The task completion time is related to path length and speed. These are weighting factors, representing the priorities of image quality, recognition accuracy, and task time, respectively. Dynamic strategy optimization module: Constructing the state space of Q-learning Action space ; Design a reward function: ， in, express Instant rewards for each moment This indicates the current image quality score. Indicates the distance between the drone and the nearest obstacle. This indicates the preset safe distance threshold. Indicates the current energy consumption. , , These are the weighting coefficients for image quality, obstacle avoidance safety, and energy consumption, respectively, representing the priorities of image quality, obstacle avoidance safety, and energy consumption. Intelligent obstacle processing module: ORB-SLAM3 is used to build 3D obstacle models in real time and calculate hazard factors. ， in, Indicates the dynamic hazard factor of the obstacle. The speed of the obstacle's movement Indicates the angle between the direction of the obstacle's movement and the drone's heading. This represents the minimum predicted distance between the drone and the obstacle. An improved APF-RRT* algorithm is used to generate obstacle avoidance trajectories. The potential field function includes: ， in, Let be the total potential field function. The potential field attracted to the target point is positively correlated with the distance to the target. This represents the potential field for image quality, which is negatively correlated with imaging performance. For a dynamic obstacle repulsive potential field, and Positive correlation These are adaptive weighting coefficients; Online collaborative control module: Real-time optimization of control parameters using the BFGS algorithm ,satisfy: ， in, Represents the control vector. Indicates the heading angle. This indicates the actual imaging quality index under the current control parameters. For ideal imaging quality indicators, This represents the reference control value, derived from global planning. This represents the regularization coefficient, which balances tracking accuracy and control smoothness.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that instructs the device to execute the UAV autonomous obstacle avoidance and dynamic path planning method based on multimodal perception and hybrid intelligent decision-making as described in any one of claims 1 to 8.

Images