Electric power unmanned aerial vehicle obstacle avoidance method and system based on multi-modal perception and reinforcement learning

Abstract

The invention discloses an electric power unmanned aerial vehicle obstacle avoidance method and system based on multi-modal perception and reinforcement learning, and the method comprises the steps: firstly starting an unmanned aerial vehicle, collecting environment and own motion data in real time, constructing an initial three-dimensional environment map after preprocessing fusion, and recognizing an obstacle in the map; based on the flight task, performing global path planning by using an improved algorithm, and generating a preliminary flight path from the starting point to the target; then, dynamically detecting the path obstacle; if no obstacle exists, working along the path; if the obstacle exists, calculating values of different obstacle avoidance actions through a reward function, and selecting an optimal action to generate a second flight path; and the unmanned aerial vehicle adjusts the attitude and speed to execute operation according to the second path and the optimal action. According to the method, the obstacle avoidance response speed of the unmanned aerial vehicle is higher, the route prediction accuracy is greatly improved, the collision risk is effectively reduced, the power inspection efficiency and safety are improved, and the method has remarkable technical advantages and application value.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric drones, and specifically relates to an electric drone obstacle avoidance method and system based on multimodal perception and reinforcement learning, which is used to improve the electric drone's autonomous obstacle avoidance capability in complex environments. Background Art

[0002] Currently, the scale of overhead power lines in distribution networks is large and continues to grow. Drone inspections, with their efficiency and flexibility, have become a crucial tool for power line inspections. However, existing transmission line inspection systems face numerous challenges that need to be addressed.

[0003] In terms of environmental perception, most systems rely on a single sensor or a simple combination of multiple sensors, making it difficult to fully and accurately perceive the complex and ever-changing transmission corridor environment. For example, relying solely on visible light cameras significantly reduces image recognition performance in poor lighting conditions (such as at night, in heavy fog, or in strong backlight), making it impossible to accurately identify obstacles. While a single lidar can acquire distance information, its semantic understanding of objects is insufficient, making it difficult to distinguish between different types of obstacles, resulting in a failure to provide comprehensive and effective environmental data for obstacle avoidance decisions. Furthermore, existing systems lack the ability to deeply integrate and collaboratively process multimodal data, failing to fully leverage the complementary advantages of multiple sensors and creating high-precision, real-time three-dimensional environmental maps. This significantly limits drones' ability to perceive obstacles in complex environments.

[0004] From the perspective of control algorithms, traditional drone control algorithms, such as those based on PID and LQR controllers, exhibit significant deficiencies when coping with complex dynamic environments. As a linear algorithm, PID controllers have poor robustness. When drones face complex situations such as electromagnetic interference, strong winds, and sudden changes in terrain, they struggle to quickly and accurately adjust their flight attitude and trajectory, which can easily lead to inaccurate or even loss of control. While LQR controllers are nonlinear and improve control accuracy to a certain extent, they rely heavily on model accuracy. The environmental factors involved in electric drone flight are highly variable, making it difficult to precisely determine model parameters, resulting in unsatisfactory control results in practical applications. Furthermore, traditional algorithms lack dynamic response speed, control bandwidth, and the ability to handle complex constraints, making them unable to meet the precise trajectory control and rapid obstacle avoidance requirements of electric drones in complex environments.

[0005] When it comes to obstacle avoidance path planning, existing technologies either focus on global path planning, failing to adjust the path in time when encountering unexpected obstacles, potentially putting the drone in danger; or they focus on local obstacle avoidance and lack consideration of the global optimal path, resulting in lengthy drone flight paths, increased energy consumption, and reduced inspection efficiency. For example, some rule-based obstacle avoidance systems are unable to flexibly adjust their avoidance strategies when faced with complex terrain and dynamic obstacles, making it difficult to achieve safe and efficient obstacle avoidance. Some existing reinforcement learning obstacle avoidance methods rely excessively on large amounts of manually labeled data for training. In actual application scenarios, due to the diversity and dynamism of the environment, the model's generalization ability is poor, making it difficult to quickly adapt to new environments and failing to meet the real-time and reliability requirements of power inspections for autonomous obstacle avoidance by drones.

[0006] At the same time, existing power drone intelligent photography systems suffer from poor image quality and insufficient adaptability to complex scenarios. Inaccurate preset waypoints lead to illogical shooting angles and distances, frequently causing overexposure and underexposure, severely impacting image quality and defect identification. Furthermore, they are unable to autonomously distinguish between different scene segments and key inspection targets, making it difficult to meet the high-quality image acquisition and precise defect identification requirements of power inspections. Furthermore, traditional edge computing terminals, limited by computing power and energy consumption, struggle to deploy complex models for efficient data processing, resulting in poor performance in intelligent photography and data processing.

[0007] In the field of drone-mounted terminals, problems exist, including insufficient positioning accuracy, low multi-drone coordination efficiency, and poor device compatibility. Traditional positioning methods struggle to achieve precise positioning in GPS-denied environments. Multi-drone collaborative task allocation lacks efficient algorithms, hindering the full utilization of multiple drone inspection capabilities. Furthermore, poorly designed airborne terminals, characterized by high power consumption and deployment costs, make them difficult to adapt to various mainstream drone platforms, limiting the overall efficiency and quality of power inspections.

[0008] These issues severely restrict the efficient and safe operation of electric drones in complex environments, necessitating innovative technologies to achieve breakthroughs. This invention, based on a multimodal perception and reinforcement learning-based electric drone obstacle avoidance system and method, along with two other related patented technologies, innovates across multiple dimensions, including environmental perception, intelligent decision-making, path planning, intelligent photography, and terminal performance. These innovations aim to address the shortcomings of existing technologies and significantly enhance the autonomous inspection capabilities and application value of electric drones. Summary of the Invention

[0009] In response to the shortcomings of the existing technology, the present invention proposes a method for generating complex dialogue questions based on a diffusion model to improve the diversity, rationality and context adaptability of dialogue questions.

[0010] An obstacle avoidance method for electric drones based on multimodal perception and reinforcement learning, including: S1: Start the drone, collect environmental data and drone motion data in real time, pre-process and fuse the environmental data and drone motion data to build a 3D environmental map and identify obstacles in the 3D environmental map; S2: Obtain the UAV's flight mission, which includes the UAV's starting point and target point. Global path planning is performed using the improved quantum wolf pack algorithm to generate a preliminary flight path from the starting point to the target point on a three-dimensional environment map. S3: Generate obstacle avoidance action decisions based on the preliminary flight path and obstacles. Specifically: if there are no obstacles on the preliminary flight path, the UAV will operate along the preliminary flight path. If there is an obstacle on the initial flight path, the value of different obstacle avoidance actions is calculated in combination with the reward function, the optimal obstacle avoidance action is selected, the second flight path is generated, and step S4 is further executed; S4: Adjust the flight attitude and speed of the UAV based on the second flight path and the optimal obstacle avoidance action, and perform operations along the second flight path; S5: During the UAV's flight operation, environmental data is continuously collected. If a new obstacle is identified, the UAV's local path is replanned, and the obstacle avoidance action and flight path are updated based on the replanned local path. The flight path includes the initial flight path or the secondary flight path.

[0011] Furthermore, step S1 is specifically as follows: The visible light camera is used to capture the RGB image sequence I(x, y, t) at a preset frame rate to obtain the fused visual features, which are expressed as: ; in, is the horizontal coordinate of the image pixel, is the vertical coordinate of the image pixel, is the timestamp, represents the fused visual features, represents the color extraction function, represents the texture feature extraction function, represents the feature transformation method extraction function; The environment point cloud model is generated by the laser radar at a preset scanning frequency, where the original point cloud set is represented as P; the effective obstacle point cloud is screened and represented as: ; in, Indicates the point cloud space coordinates, puav indicates the current position coordinates of the drone, Indicates the maximum detection distance of the laser radar, is the Euclidean distance calculation, It is the screened effective obstacle point cloud; The UAV’s motion state vector A is outputted through the inertial measurement unit (IMU) at a preset sampling rate. The sensor data including the fused visual features, effective obstacle point cloud and UAV motion state vector is preprocessed as follows: Time synchronization: Use hardware triggering to align clocks and compensate for delays, expressed as: ; in, represents the raw sensor timestamp, Indicates the timestamp after synchronization. represents the delay compensation amount, It means that the mean is 0 and the variance is Gaussian distribution; Spatial calibration: The coordinate transformation between sensors is established through the calibration plate, which is expressed as: ; in, Represents the transformation matrix from the lidar coordinate system to the camera coordinate system, R represents the 3×3 rotation matrix, t represents the 3D translation vector, represents the set of real numbers; The Kalman filter algorithm is used to fuse the above processed data and construct a three-dimensional environmental map.

[0012] Furthermore, the Kalman filter algorithm is used to fuse the processed data and construct a three-dimensional environment map, specifically: State prediction estimates the dynamic changes of the environment state based on the UAV motion model and predicts the obstacle position and motion trajectory, which can be expressed as: ; Where k represents the current time step index; k-1 represents the previous time step index; X represents the system state vector, which contains the drone's position, speed, attitude, and environmental obstacle information; represents the state vector at time k-1, represents the state vector at time k; Represents the state transition matrix at time k, and predicts the state change based on the UAV kinematic model; represents the control input matrix; represents the control input vector; P represents the state covariance matrix, which quantifies the uncertainty of the state estimation; represents the state covariance at time k-1; represents the prior state covariance at time k; represents the process noise covariance matrix; T represents the matrix transpose; Observation update, fusing sensor data, and correcting errors in state prediction; expressed as: ; in, represents the Kalman gain at time k, which is the weight of the sensor observation; Represents the observation matrix, which is used to establish the mapping relationship from the state space to the sensor observation space; Represents the observation noise covariance matrix; -1 represents matrix inversion; Represents sensor observations, including fused visual features, valid obstacle point clouds, and the UAV's motion state vector; represents the posterior state estimate at time k; Generate a 3D environment map and identify obstacles, expressed as: ; Among them, Map represents the three-dimensional environment map function, (x, y, z) represents the three-dimensional space coordinates, represents the sensor weight coefficient, represents the voxel unit, Represents the occupation state evaluation function, according to the state vector The occupancy probability of each voxel is calculated based on the obstacle information in , where N represents the total number of voxels and i is the voxel index.

[0013] Furthermore, in step S2, the global path planning is performed using the improved quantum wolf pack algorithm, specifically: Initialization: Starting point of the drone and target point As the spatial boundary, N wolf pack individual positions are randomly generated, expressed as: ; in, represents the wolf individual index; represents the initial position of the i-th wolf, each position represents a candidate path; λ represents a random number uniformly distributed in the interval [0,1], used for path point coordinate calculation; Indicates the size of the wolf pack; Quantum coding and fitness evaluation: The superposition state of the quantum bit encoding path solution is expressed as:

[0014] in, represents the quantum state of the i-th wolf; and is the complex probability amplitude, satisfying ;|0> and |1> represent the basic quantum states; The quality of the path solution is evaluated by the fitness function, which is expressed as:

[0015] in, represents the fitness value of the i-th path solution; is the weight coefficient and satisfies =1; Path length cost calculation: ; in, represents the path length cost; represents the coordinates of the kth node in the path, represents the coordinates of the k-1th node on the path; M represents the total number of nodes on the path; Obstacle avoidance cost calculation, expressed as: ; in, represents the obstacle avoidance cost, represents the position of the jth obstacle; Indicates the total number of obstacles; Time consumption cost calculation, expressed as: ; in, Indicates the time consumption cost; Indicates the maximum flight speed of the drone; Global path planning: The quantum revolving door update is expressed as: ; in, represents the quantum rotating gate matrix; Indicates the rotation angle, according to the current optimal solution Dynamic adjustment; new represents the updated quantum state; Through the path nodes Affecting quantum rotation angle The adjustment strategy is as follows: the longer the path The smaller; Simulate the pursuit behavior, expressed as: ; in, Indicates the new position after the pursuit behavior; represents the current optimal solution; represents the pursuit step length coefficient; represents the probability of quantum state collapse; By adjusting the pursuit step length Controls the search speed when approaching obstacles, specifically When the value increases reduce; Simulate siege behavior, expressed as: ; in, Indicates the new position after the siege behavior; represents the set of neighboring wolf locations; represents the number of neighboring wolves; represents the quantum perturbation factor; represents the quantum random perturbation term; By controlling The value adjusts the path convergence speed, specifically as Increase Increase to accelerate convergence; Fitness update and comparison: recalculate after each round of update Value, compare and update the global optimal solution ; Optimal solution output: When the number of iterations reaches the preset value or the fitness changes When, among them, is the convergence threshold, and the global optimal path is output, that is, the global optimal solution ; Local obstacle avoidance path adjustment process: When an obstacle is detected, a local search area is generated, which is expressed as: ; ; in, represents the local search area; Indicates the location of the obstacle; represents the search radius, is the minimum safe distance, is the expansion coefficient; Quantum random walk, expressed as: ; in, Indicates the position of the adjusted path point; represents a random step size; represents the random perturbation of the quantum state; pass Dynamic adjustment, specifically The larger the value The smaller it is to avoid collisions; Obstacle avoidance driving path optimization: exist Re-evaluate obstacle avoidance costs within the region , increase the area near the obstacle The weight coefficient ; Output bypasses obstacles The local optimal path.

[0016] Furthermore, in step S3, if there is an obstacle on the preliminary flight path, the value of different obstacle avoidance actions is calculated in combination with the reward function, and the optimal obstacle avoidance action is selected, specifically: The DQN network is used to encode the drone state and select actions; specifically: The DQN network receives three-dimensional environment map information and encodes it into a state vector, which is expressed as:

[0017] in, represents the environmental state vector at time t; Represents the current three-dimensional environment map; Indicates the current position coordinates of the drone; Represents the current velocity vector of the drone; Represents a dynamic obstacle set; represents the encoding function; Defining the obstacle avoidance action space , contains discrete control instructions, expressed as: ; in, Indicates the yaw angle adjustment amount; Indicates the pitch angle adjustment amount; Indicates the speed change; pass -greedy strategy selects actions, expressed as: ; in, Represents the action value output by the DQN network; Represents the network weight parameters; is the exploration rate; The reward function calculation unit of the DQN network generates a reward signal , expressed as: ; ; ; ; ; in, Represents the obstacle avoidance result reward; represents the route smoothness penalty, hour, Increased from 0.1 to 0.5; represents energy consumption penalty; Experience replay and network training, specifically: Quadruple storage: After the decision is executed, the experience samples are generated and stored in the experience replay storage unit: ; in, represents the environmental state vector at time t; represents the action performed at time t; Rt represents the reward value calculated by the reward function at time t; Indicates the new state after executing the action; MemoryBuffer represents the experience playback storage unit; Random sampling training: When the cumulative amount of experience playback unit data exceeds the threshold, batch sampling training is performed, which is expressed as: ; Where E represents the mathematical expectation operator, which averages the samples in the sampling batch; Represents a single piece of experience data sampled from MemoryBuffer; s is the sampling state, For sampling action, is the corresponding reward value, s' is the next state; γ is the discount factor, which is used to adjust the weight of future rewards; represents the target network's maximum action value prediction for the next state s'; represents the target network parameters; Represents the value prediction of action a in state s; Network parameter update: Update online network parameters through back propagation algorithm: ; Where θ represents the current parameter vector of the DQN network; represents the learning rate; Represents the gradient calculation operator for the parameter θ; The decision output unit of the DQN network performs the following operations: Action value evaluation: for each candidate action in action set A , calculate the corresponding value assessment value, expressed as: ; in, Represents the current environment state vector; represents the jth candidate obstacle avoidance action; θ represents the DQN network parameters; Indicates action In state The following valuation value; Optimal action selection: Select the action with the highest value as the execution instruction: expressed as

[0018] in, represents the optimal obstacle avoidance action selected; represents the operation of taking the index j that maximizes the Q value;

[0019] Control command conversion: Convert the action command into the flight control system executable format, expressed as: ; Among them, ControlCmd represents the control instruction that is finally output; Indicates the adjusted pitch angle; Indicates the adjusted yaw angle; Indicates the adjusted speed value; Indicates the current actual pitch angle; Indicates the current actual yaw angle; Indicates the current actual speed value; Indicates the pitch angle adjustment amount in the optimal action; represents the yaw angle adjustment amount in the optimal action; Indicates the speed change in the optimal action.

[0020] An electric drone obstacle avoidance system based on multimodal perception and reinforcement learning, used to implement the above-mentioned electric drone obstacle avoidance method based on multimodal perception and reinforcement learning, comprising: Multimodal perception module, which collects and outputs environmental information; A reinforcement learning decision module is connected to the multimodal perception module to receive environmental information and output action instructions; A bionic swarm intelligent optimization module, which interfaces with the multimodal perception module, performs path planning based on environmental information, and outputs path information to the reinforcement learning decision module; The flight control module is connected to the reinforcement learning decision module and the bionic swarm intelligent optimization module respectively, receives control instructions, and outputs the real-time flight status of the UAV to the multimodal perception module for updating environmental information.

[0021] The beneficial effects of the present invention are:

[0022] This invention combines multimodal perception with reinforcement learning and bionic swarm intelligence optimization, overcoming the limitations of traditional obstacle avoidance systems. Compared to existing technologies, it enables drones to avoid obstacles more quickly and achieve significantly improved route prediction accuracy in complex and dynamic environments, effectively reducing collision risks and improving the efficiency and safety of power inspections. It offers significant technical advantages and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0024] The following describes the embodiments of the present invention through specific examples. Those skilled in the art will readily understand the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through various other specific embodiments, and the details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention.

[0025] Reference Figure 1 , an electric UAV obstacle avoidance method based on multimodal perception and reinforcement learning, including: S1: Start the drone, collect environmental data and drone motion data in real time, pre-process and fuse the environmental data and drone motion data to build a 3D environmental map and identify obstacles in the 3D environmental map; S2: Obtain the UAV's flight mission, which includes the UAV's starting point and target point. Global path planning is performed using the improved quantum wolf pack algorithm to generate a preliminary flight path from the starting point to the target point on a three-dimensional environment map. S3: Generate obstacle avoidance action decisions based on the preliminary flight path and obstacles. Specifically: if there are no obstacles on the preliminary flight path, the UAV will operate along the preliminary flight path. If there is an obstacle on the initial flight path, the value of different obstacle avoidance actions is calculated in combination with the reward function, the optimal obstacle avoidance action is selected, the second flight path is generated, and step S4 is further executed; S4: Adjust the flight attitude and speed of the UAV based on the second flight path and the optimal obstacle avoidance action, and perform operations along the second flight path; S5: During the UAV's flight operation, environmental data is continuously collected. If a new obstacle is identified, the UAV's local path is replanned, and the obstacle avoidance action and flight path are updated based on the replanned local path. The flight path includes the initial flight path or the secondary flight path.

[0026] Furthermore, step S1 is specifically as follows: The visible light camera is used to capture the RGB image sequence I(x, y, t) at a preset frame rate to obtain the fused visual features, which are expressed as: ; in, is the horizontal coordinate of the image pixel, is the vertical coordinate of the image pixel, is the timestamp, represents the fused visual features, represents the color extraction function, represents the texture feature extraction function, represents the feature transformation method extraction function; The environment point cloud model is generated by the laser radar at a preset scanning frequency, where the original point cloud set is represented as P; the effective obstacle point cloud is screened and represented as: ; in, Indicates the point cloud space coordinates, puav indicates the current position coordinates of the drone, Indicates the maximum detection distance of the laser radar, is the Euclidean distance calculation, It is the screened effective obstacle point cloud; The UAV’s motion state vector A is outputted through the inertial measurement unit (IMU) at a preset sampling rate. The sensor data including the fused visual features, effective obstacle point cloud and UAV motion state vector is preprocessed as follows: Time synchronization: Use hardware triggering to align clocks and compensate for delays, expressed as: ; in, represents the raw sensor timestamp, Indicates the timestamp after synchronization. represents the delay compensation amount, It means that the mean is 0 and the variance is Gaussian distribution; Spatial calibration: The coordinate transformation between sensors is established through the calibration plate, which is expressed as: ; in, Represents the transformation matrix from the lidar coordinate system to the camera coordinate system, R represents the 3×3 rotation matrix, t represents the 3D translation vector, represents the set of real numbers; The Kalman filter algorithm is used to fuse the above processed data and construct a three-dimensional environmental map.

[0027] Furthermore, the Kalman filter algorithm is used to fuse the processed data and construct a three-dimensional environment map, specifically: State prediction estimates the dynamic changes of the environment state based on the UAV motion model and predicts the obstacle position and motion trajectory, which can be expressed as: ; Where k represents the current time step index; k-1 represents the previous time step index; X represents the system state vector, which contains the drone's position, speed, attitude, and environmental obstacle information; represents the state vector at time k-1, represents the state vector at time k; Represents the state transition matrix at time k, and predicts the state change based on the UAV kinematic model; represents the control input matrix; represents the control input vector; P represents the state covariance matrix, which quantifies the uncertainty of the state estimation; represents the state covariance at time k-1; represents the prior state covariance at time k; represents the process noise covariance matrix; T represents the matrix transpose; Observation update, fusing sensor data, and correcting errors in state prediction; expressed as: ; in, represents the Kalman gain at time k, which is the weight of the sensor observation; Represents the observation matrix, which is used to establish the mapping relationship from the state space to the sensor observation space; Represents the observation noise covariance matrix; -1 represents matrix inversion; Represents sensor observations, including fused visual features, valid obstacle point clouds, and the UAV's motion state vector; represents the posterior state estimate at time k; Generate a 3D environment map and identify obstacles, expressed as: ; Among them, Map represents the three-dimensional environment map function, (x, y, z) represents the three-dimensional space coordinates, represents the sensor weight coefficient, represents the voxel unit, Represents the occupation state evaluation function, according to the state vector The occupancy probability of each voxel is calculated based on the obstacle information in , where N represents the total number of voxels and i is the voxel index.

[0028] Furthermore, in step S2, the global path planning is performed using the improved quantum wolf pack algorithm, specifically: Initialization: Starting point of the drone and target point As the spatial boundary, N wolf pack individual positions are randomly generated, expressed as: ; in, represents the wolf individual index; represents the initial position of the i-th wolf, each position represents a candidate path; λ represents a random number uniformly distributed in the interval [0,1], used for path point coordinate calculation; Indicates the size of the wolf pack; Quantum coding and fitness evaluation: The superposition state of the quantum bit encoding path solution is expressed as:

[0029] in, represents the quantum state of the i-th wolf; and is the complex probability amplitude, satisfying ;|0> and |1> represent the basic quantum states; The quality of the path solution is evaluated by the fitness function, which is expressed as:

[0030] in, represents the fitness value of the i-th path solution; is the weight coefficient and satisfies =1; Path length cost calculation: ; in, represents the path length cost; represents the coordinates of the kth node in the path, represents the coordinates of the k-1th node on the path; M represents the total number of nodes on the path; Obstacle avoidance cost calculation, expressed as: ; in, represents the obstacle avoidance cost, represents the position of the jth obstacle; Indicates the total number of obstacles; Time consumption cost calculation, expressed as: ; in, Indicates the time consumption cost; Indicates the maximum flight speed of the drone; Global path planning: The quantum revolving door update is expressed as: ; in, represents the quantum rotating gate matrix; Indicates the rotation angle, according to the current optimal solution Dynamic adjustment; new represents the updated quantum state; Through the path nodes Affecting quantum rotation angle The adjustment strategy is as follows: the longer the path The smaller; Simulate the pursuit behavior, expressed as: ; in, Indicates the new position after the pursuit behavior; represents the current optimal solution; represents the pursuit step length coefficient; represents the probability of quantum state collapse; By adjusting the pursuit step length Controls the search speed when approaching obstacles, specifically When the value increases reduce; Simulate siege behavior, expressed as: ; in, Indicates the new position after the siege behavior; represents the set of neighboring wolf locations; represents the number of neighboring wolves; represents the quantum perturbation factor; represents the quantum random perturbation term; By controlling The value adjusts the path convergence speed, specifically as Increase Increase to accelerate convergence; Fitness update and comparison: recalculate after each round of update Value, compare and update the global optimal solution ; Optimal solution output: When the number of iterations reaches the preset value or the fitness changes When, among them, is the convergence threshold, and the global optimal path is output, that is, the global optimal solution ; Local obstacle avoidance path adjustment process: When an obstacle is detected, a local search area is generated, which is expressed as: ; ; in, represents the local search area; Indicates the location of the obstacle; represents the search radius, is the minimum safe distance, is the expansion coefficient; Quantum random walk, expressed as: ; in, Indicates the position of the adjusted path point; represents a random step size; represents the random perturbation of the quantum state; pass Dynamic adjustment, specifically The larger the value The smaller it is to avoid collisions; Obstacle avoidance driving path optimization: exist Re-evaluate obstacle avoidance costs within the region , increase the area near the obstacle The weight coefficient ; Output bypasses obstacles The local optimal path.

[0031] Furthermore, in step S3, if there is an obstacle on the preliminary flight path, the value of different obstacle avoidance actions is calculated in combination with the reward function, and the optimal obstacle avoidance action is selected, specifically: The DQN network is used to encode the drone state and select actions; specifically: The DQN network receives three-dimensional environment map information and encodes it into a state vector, which is expressed as:

[0032] in, represents the environmental state vector at time t; Represents the current three-dimensional environment map; Indicates the current position coordinates of the drone; Represents the current velocity vector of the drone; Represents a dynamic obstacle set; represents the encoding function; Defining the obstacle avoidance action space , contains discrete control instructions, expressed as: ; in, Indicates the yaw angle adjustment amount; Indicates the pitch angle adjustment amount; Indicates the speed change; pass -greedy strategy selects actions, expressed as: ; in, Represents the action value output by the DQN network; Represents the network weight parameters; is the exploration rate; The reward function calculation unit of the DQN network generates a reward signal , expressed as: ; ; ; ; ; in, Represents the obstacle avoidance result reward; represents the route smoothness penalty, hour, Increased from 0.1 to 0.5; represents energy consumption penalty; Experience replay and network training, specifically: Quadruple storage: After the decision is executed, the experience samples are generated and stored in the experience replay storage unit: ; in, represents the environmental state vector at time t; represents the action performed at time t; Rt represents the reward value calculated by the reward function at time t; Indicates the new state after executing the action; MemoryBuffer represents the experience playback storage unit; Random sampling training: When the cumulative amount of experience playback unit data exceeds the threshold, batch sampling training is performed, which is expressed as: ; Where E represents the mathematical expectation operator, which averages the samples in the sampling batch; Represents a single piece of experience data sampled from MemoryBuffer; s is the sampling state, For sampling action, is the corresponding reward value, s' is the next state; γ is the discount factor, which is used to adjust the weight of future rewards; represents the target network's maximum action value prediction for the next state s'; represents the target network parameters; Represents the value prediction of action a in state s; Network parameter update: Update online network parameters through back propagation algorithm: ; Where θ represents the current parameter vector of the DQN network; represents the learning rate; Represents the gradient calculation operator for the parameter θ; The decision output unit of the DQN network performs the following operations: Action value evaluation: for each candidate action in action set A , calculate the corresponding value assessment value, expressed as: ; in, Represents the current environment state vector; represents the jth candidate obstacle avoidance action; θ represents the DQN network parameters; Indicates action In state The following valuation value; Optimal action selection: Select the action with the highest value as the execution instruction: expressed as

[0033] in, represents the optimal obstacle avoidance action selected; represents the operation of taking the index j that maximizes the Q value;

[0034] Control command conversion: Convert the action command into the flight control system executable format, expressed as: ; Among them, ControlCmd represents the control instruction that is finally output; Indicates the adjusted pitch angle; Indicates the adjusted yaw angle; Indicates the adjusted speed value; Indicates the current actual pitch angle; Indicates the current actual yaw angle; Indicates the current actual speed value; Indicates the pitch angle adjustment amount in the optimal action; represents the yaw angle adjustment amount in the optimal action; Indicates the speed change in the optimal action.

[0035] An electric drone obstacle avoidance system based on multimodal perception and reinforcement learning, used to implement the above-mentioned electric drone obstacle avoidance method based on multimodal perception and reinforcement learning, comprising: Multimodal perception module, which collects and outputs environmental information; A reinforcement learning decision module is connected to the multimodal perception module to receive environmental information and output action instructions; A bionic swarm intelligent optimization module, which interfaces with the multimodal perception module, performs path planning based on environmental information, and outputs path information to the reinforcement learning decision module; The flight control module is connected to the reinforcement learning decision module and the bionic swarm intelligent optimization module respectively, receives control instructions, and outputs the real-time flight status of the UAV to the multimodal perception module for updating environmental information.

[0036] The present invention has been described in detail above with reference to the embodiments of the accompanying drawings. A person skilled in the art can make various modifications to the present invention based on the above description. Therefore, certain details in the embodiments should not be construed as limiting the present invention. The scope of protection of the present invention shall be determined by the scope defined by the appended claims.

Claims

1. A method for electric drone obstacle avoidance based on multimodal perception and reinforcement learning, characterized in that: include: S1: Start the drone, collect environmental data and drone motion data in real time, pre-process and fuse the environmental data and drone motion data to build a 3D environmental map and identify obstacles in the 3D environmental map; S2: Obtain the UAV's flight mission, which includes the UAV's starting point and target point. Global path planning is performed using the improved quantum wolf pack algorithm to generate a preliminary flight path from the starting point to the target point on a three-dimensional environment map. S3: Generate obstacle avoidance action decisions based on the preliminary flight path and obstacles. Specifically, if there are no obstacles on the preliminary flight path, the UAV will operate along the preliminary flight path. If there is an obstacle on the initial flight path, the value of different obstacle avoidance actions is calculated in combination with the reward function, the optimal obstacle avoidance action is selected, the second flight path is generated, and step S4 is further executed; S4: Adjust the flight attitude and speed of the UAV based on the second flight path and the optimal obstacle avoidance action, and operate along the second flight path; S5: During the UAV's flight operation, environmental data is continuously collected. If a new obstacle is identified, the UAV's local path is replanned, and the obstacle avoidance action and flight path are updated based on the replanned local path. The flight path includes the initial flight path or the secondary flight path.

2. The electric drone obstacle avoidance method based on multimodal perception and reinforcement learning according to claim 1 is characterized in that: Step S1 is specifically as follows: The visible light camera is used to capture the RGB image sequence I(x, y, t) at a preset frame rate to obtain the fused visual features, which are expressed as: ； in, is the horizontal coordinate of the image pixel, is the vertical coordinate of the image pixel, is the timestamp, represents the fused visual features, represents the color extraction function, represents the texture feature extraction function, represents the feature transformation method extraction function; The environment point cloud model is generated by the laser radar at a preset scanning frequency, where the original point cloud set is represented as P; the effective obstacle point cloud is screened and represented as: ； in, Indicates the point cloud space coordinates, puav indicates the current position coordinates of the drone, Indicates the maximum detection distance of the laser radar, is the Euclidean distance calculation, It is the screened effective obstacle point cloud; The UAV’s motion state vector A is outputted through the inertial measurement unit (IMU) at a preset sampling rate. The sensor data including the fused visual features, effective obstacle point cloud and UAV motion state vector is preprocessed as follows: Time synchronization: Use hardware triggering to align clocks and compensate for delays, expressed as: ； in, represents the raw sensor timestamp, Indicates the timestamp after synchronization. represents the delay compensation amount, It means that the mean is 0 and the variance is Gaussian distribution; Spatial calibration: The coordinate transformation between sensors is established through the calibration plate, which is expressed as: ； in, Represents the transformation matrix from the lidar coordinate system to the camera coordinate system, R represents the 3×3 rotation matrix, t represents the 3D translation vector, represents the set of real numbers; The Kalman filter algorithm is used to fuse the above processed data and construct a three-dimensional environmental map.

3. The electric drone obstacle avoidance method based on multimodal perception and reinforcement learning according to claim 1 is characterized in that: The Kalman filter algorithm is used to fuse the above processed data and construct a three-dimensional environment map, specifically: State prediction, based on the UAV motion model, estimates the dynamic changes of the environment state and predicts the obstacle position and motion trajectory, which can be expressed as: ； Where k represents the current time step index; k-1 represents the previous time step index; X represents the system state vector, which contains the drone's position, speed, attitude, and environmental obstacle information; represents the state vector at time k-1, represents the state vector at time k; Represents the state transition matrix at time k, and predicts the state change based on the UAV kinematic model; represents the control input matrix; represents the control input vector; P represents the state covariance matrix, which quantifies the uncertainty of the state estimation; represents the state covariance at time k-1; represents the prior state covariance at time k; represents the process noise covariance matrix; T represents the matrix transpose; Observation update, fusing sensor data, and correcting errors in state prediction; expressed as: ； in, represents the Kalman gain at time k, that is, the weight of the sensor observation; Represents the observation matrix, which is used to establish the mapping relationship from the state space to the sensor observation space; Represents the observation noise covariance matrix; -1 represents matrix inversion; Represents sensor observations, including fused visual features, valid obstacle point clouds, and the UAV's motion state vector; represents the posterior state estimate at time k; Generate a 3D environment map and identify obstacles, expressed as: ； Among them, Map represents the three-dimensional environment map function, (x, y, z) represents the three-dimensional space coordinates, represents the sensor weight coefficient, represents the voxel unit, Represents the occupation state evaluation function, according to the state vector The occupancy probability of each voxel is calculated based on the obstacle information in , where N represents the total number of voxels and i is the voxel index.

4. The electric drone obstacle avoidance method based on multimodal perception and reinforcement learning according to claim 3 is characterized in that: In step S2, the global path planning is performed using the improved quantum wolf pack algorithm, specifically: Initialization: Starting point of the drone and target point As the spatial boundary, N wolf pack individual positions are randomly generated, expressed as: ； in, represents the individual wolf index; represents the initial position of the i-th wolf, each position represents a candidate path; λ represents a random number uniformly distributed in the interval [0,1], used for path point coordinate calculation; Indicates the size of the wolf pack; Quantum coding and fitness evaluation: The superposition state of the quantum bit encoding path solution is expressed as:

5. Among them, represents the quantum state of the i-th wolf; and is the complex probability amplitude, satisfying ;|0> and |1> represent the basic quantum states; The quality of the path solution is evaluated by the fitness function, which is expressed as:

6. Among them, represents the fitness value of the i-th path solution; is the weight coefficient and satisfies =1; Path length cost calculation: ； in, represents the path length cost; represents the coordinates of the kth node in the path, represents the coordinates of the k-1th node on the path; M represents the total number of nodes on the path; Obstacle avoidance cost calculation, expressed as: ； in, represents the obstacle avoidance cost, represents the position of the jth obstacle; Indicates the total number of obstacles; Time consumption cost calculation, expressed as: ； in, Indicates the time consumption cost; Indicates the maximum flight speed of the drone; Global path planning: The quantum revolving door update is expressed as: ； in, represents the quantum rotating gate matrix; Indicates the rotation angle, according to the current optimal solution Dynamic adjustment; new represents the updated quantum state; Through the path node Affecting quantum rotation angle The adjustment strategy is as follows: the longer the path The smaller; Simulate the pursuit behavior, expressed as: ； in, Indicates the new position after the pursuit behavior; represents the current optimal solution; represents the pursuit step length coefficient; represents the probability of quantum state collapse; By adjusting the pursuit step length Controls the search speed when approaching obstacles, specifically When the value increases reduce; Simulate siege behavior, expressed as: ； in, Indicates the new position after the siege behavior; represents the set of neighboring wolf locations; represents the number of neighboring wolves; represents the quantum perturbation factor; represents the quantum random perturbation term; By controlling The value adjusts the path convergence speed, specifically as Increase Increase to accelerate convergence; Fitness update and comparison: recalculate after each round of update Value, compare and update the global optimal solution ; Optimal solution output: When the number of iterations reaches the preset value or the fitness changes When, among them, is the convergence threshold, and the global optimal path is output, that is, the global optimal solution ; Local obstacle avoidance path adjustment process: When an obstacle is detected, a local search area is generated, which is expressed as: ； ； in, represents the local search area; Indicates the location of the obstacle; represents the search radius, is the minimum safe distance, is the expansion coefficient; Quantum random walk, expressed as: ； in, Indicates the position of the waypoint after adjustment; represents a random step size; represents the random perturbation of the quantum state; pass Dynamic adjustment, specifically The larger the value The smaller it is to avoid collisions; Obstacle avoidance driving path optimization: exist Re-evaluate obstacle avoidance costs within the region , increase the area near the obstacle The weight coefficient ; Output bypasses obstacles The local optimal path.

7. The electric drone obstacle avoidance method based on multimodal perception and reinforcement learning according to claim 4 is characterized in that: In step S3, if there is an obstacle on the initial flight path, the value of different obstacle avoidance actions is calculated based on the reward function, and the optimal obstacle avoidance action is selected, specifically: The DQN network is used to encode the drone state and select actions; specifically: The DQN network receives three-dimensional environment map information and encodes it into a state vector, which is expressed as:

8. Among them, represents the environmental state vector at time t; Represents the current three-dimensional environment map; Indicates the current position coordinates of the drone; Represents the current velocity vector of the drone; Represents a dynamic obstacle set; represents the encoding function; Defining the obstacle avoidance action space , contains discrete control instructions, expressed as: ； in, Indicates the yaw angle adjustment amount; Indicates the pitch angle adjustment amount; Indicates the speed change; pass -greedy strategy selects actions, expressed as: ； in, Represents the action value output by the DQN network; Represents the network weight parameters; is the exploration rate; The reward function calculation unit of the DQN network generates a reward signal , expressed as: ； ； ； ； ； in, Represents the obstacle avoidance result reward; represents the route smoothness penalty, hour, Increased from 0.1 to 0.5; represents energy consumption penalty; Experience replay and network training, specifically: Quadruple storage: After the decision is executed, the experience samples are generated and stored in the experience replay storage unit: ； in, represents the environmental state vector at time t; represents the action performed at time t; Rt represents the reward value calculated by the reward function at time t; Indicates the new state after executing the action; MemoryBuffer represents the experience playback storage unit; Random sampling training: When the cumulative amount of experience playback unit data exceeds the threshold, batch sampling training is performed, which is expressed as: ； Where E represents the mathematical expectation operator, which averages the samples in the sampling batch; Represents a single piece of experience data sampled from MemoryBuffer; s is the sampling state, For sampling action, is the corresponding reward value, s' is the next state; γ is the discount factor, which is used to adjust the weight of future rewards; represents the target network's maximum action value prediction for the next state s'; represents the target network parameters; Represents the value prediction of action a in state s; Network parameter update: Update online network parameters through back propagation algorithm: ； Where θ represents the current parameter vector of the DQN network; represents the learning rate; Represents the gradient calculation operator for the parameter θ; The decision output unit of the DQN network performs the following operations: Action value evaluation: for each candidate action in the action set A , calculate the corresponding value assessment value, expressed as: ； in, Represents the current environment state vector; represents the jth candidate obstacle avoidance action; θ represents the DQN network parameters; Indicates action In state The following valuation value; Optimal action selection: Select the action with the highest value as the execution instruction: expressed as 9. Among them, represents the optimal obstacle avoidance action selected; represents the operation of taking the index j that maximizes the Q value; Control command conversion: Convert the action command into the flight control system executable format, expressed as: ; Among them, ControlCmd represents the control instruction that is finally output; Indicates the adjusted pitch angle; Indicates the adjusted yaw angle; Indicates the adjusted speed value; Indicates the current actual pitch angle; Indicates the current actual yaw angle; Indicates the current actual speed value; Indicates the pitch angle adjustment amount in the optimal action; represents the yaw angle adjustment amount in the optimal action; Indicates the speed change in the optimal action.

10. An electric drone obstacle avoidance system based on multimodal perception and reinforcement learning, used to implement the electric drone obstacle avoidance method based on multimodal perception and reinforcement learning as described in any one of claims 1 to 5, characterized in that: include: Multimodal perception module, which collects and outputs environmental information; A reinforcement learning decision module is connected to the multimodal perception module to receive environmental information and output action instructions; A bionic swarm intelligent optimization module, which interfaces with the multimodal perception module, performs path planning based on environmental information, and outputs path information to the reinforcement learning decision module; The flight control module is connected to the reinforcement learning decision module and the bionic swarm intelligent optimization module respectively, receives control instructions, and outputs the real-time flight status of the UAV to the multimodal perception module for updating environmental information.

Images