Method and system for dynamically planning inspection path of unmanned aerial vehicle based on reinforcement learning

Abstract

The invention relates to the technical field of unmanned aerial vehicle autonomous control, and discloses an unmanned aerial vehicle inspection path dynamic planning method and system based on reinforcement learning, and the method comprises the steps: constructing a three-dimensional voxel grid through sensor data, and updating the observation confidence; constructing an aerodynamic impedance tensor representing motion damping characteristics based on the local wind field and turbulence intensity, and calculating aerodynamic potential energy according to the aerodynamic impedance tensor; performing spatial convolution on the original viewpoint information entropy by using an anisotropic information entropy convolution kernel determined by a pneumatic impedance tensor, and generating an effective information potential energy field after pneumatic fuzzy modulation; acquiring a coupling coefficient through a pre-trained reinforcement learning model, and performing weighted synthesis on the pneumatic potential energy and the effective information potential energy field to obtain a total navigation potential field; and calculating the potential field gradient and dynamically adjusting the prediction step length in combination with the pneumatic potential energy gradient modulus length so as to generate a speed control instruction. According to the invention, anisotropic modulation is carried out on information gain through aerodynamic characteristics, and adaptive optimization of flight safety, energy efficiency and inspection efficiency in a complex airflow environment is realized.

Description

Technical Field

[0001] This invention relates to the field of autonomous control technology for unmanned aerial vehicles (UAVs), specifically to a method and system for dynamic planning of UAV inspection paths based on reinforcement learning. Background Technology

[0002] With the rapid development of UAV technology, its application in complex scenarios such as power line inspection, building surveying, and disaster relief is becoming increasingly widespread. In these tasks, UAVs typically need to autonomously plan paths in unknown or semi-unknown environments to construct 3D models of the environment or complete the observation of specific targets in the shortest possible time and energy consumption. Traditional path planning methods, such as the sampling-based Rapid Expanding Random Tree (RRT) algorithm or the search-based A* algorithm, mainly focus on geometric obstacle avoidance, that is, abstracting the environment into a binary occupied and unoccupied space. The planned paths often only guarantee geometric collision-free operation, while ignoring the impact of the complex aerodynamic environment on the UAV's dynamic characteristics during flight.

[0003] In real-world 3D inspection scenarios, especially involving high-rise buildings, canyons, or offshore facilities, the local micro-meteorological environment is complex and variable, with non-uniform wind fields and turbulence. Most existing planning schemes treat wind fields as a simple, constant disturbance or completely ignore them, leading to planned paths that may traverse strong headwinds or highly turbulent areas. This not only significantly increases the energy consumption of the UAV and shortens mission endurance, but in severe cases, it can even cause flight accidents due to airflow disturbances exceeding control limits. Although some research has begun to incorporate energy consumption models into path planning, these models are often based on static global wind field assumptions and lack detailed modeling of local aerodynamic impedance characteristics, making it difficult to make real-time optimal decisions in dynamically changing and complex flow fields.

[0004] Furthermore, in Next-Best-View planning research aimed at information acquisition, existing mainstream methods typically calculate the information value of a viewpoint solely based on maximizing information entropy or exploration gain. However, this strategy of simply pursuing information gain often disconnects from the constraints of the physical environment. For example, a viewpoint with high information reward may be located in a region of strong turbulence or require significant maneuvering costs to reach. Due to the lack of a mechanism to couple the damping characteristics of the physical environment with the gain characteristics of the information field within the same manifold space, existing technologies struggle to find a dynamic balance between acquiring high-value information and avoiding high aerodynamic risks. Simultaneously, traditional artificial potential field methods or rule-based control strategies typically have fixed parameters (such as gravitational and repulsive gain, prediction step size, etc.) or rely solely on simple rules for adjustment. When faced with drastic changes in environmental gradients, these methods are prone to path oscillations, getting trapped in local minima, or slow progress in stable regions, making them unsuitable for the efficient inspection requirements in unstructured environments. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for dynamic planning of UAV inspection paths based on reinforcement learning, which solves the problem that it is difficult to balance flight safety, energy efficiency and information acquisition efficiency when UAVs are carrying out inspection tasks in complex airflow disturbance environments.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The first aspect of the present invention provides a dynamic planning method for UAV inspection paths based on reinforcement learning. This method mainly includes the following steps: First, a three-dimensional voxel grid is constructed using data collected by airborne sensing components, and voxel attributes are updated in real time to provide environmental basis data for viewpoint entropy calculation. The voxel attributes include occupancy probability and observation confidence. Second, an aerodynamic impedance tensor characterizing the damping properties of spatial motion is constructed based on airspeed and inertial data. Aerodynamic potential energy is calculated using this aerodynamic impedance tensor through quadratic form operations. This step transforms the vectorized wind field data into a second-order tensor field, thereby quantifying the distribution of motion drag in different directions in space.

[0007] Furthermore, this method calculates the original viewpoint information entropy of voxels within a 3D voxel grid through field-of-view geometric analysis, and then spatially convolves the original viewpoint information entropy using an anisotropic information entropy convolution kernel defined by the aerodynamic impedance tensor characteristics to generate an effective information potential energy field modulated by aerodynamic fuzziness. The core principle of this step lies in using the damping characteristics of the physical environment to spatially filter the information gain field, suppressing the information gain in high-damped regions at the planning level, thereby guiding the path to naturally avoid highly turbulent or strong headwind areas.

[0008] Subsequently, the UAV and environmental state features are collected and input into the pre-trained reinforcement learning policy model to obtain the coupling coefficient and prediction time-domain adjustment factor. The aerodynamic potential energy and the effective information potential energy field are linearly weighted and synthesized according to the coupling coefficient to obtain the total navigation potential field. Finally, the gradient vector of the total navigation potential field is calculated using the gradient difference operator, and the prediction step size is dynamically adjusted using the gradient magnitude of the aerodynamic potential energy combined with the prediction time-domain adjustment factor to generate a velocity control command along the negative gradient direction of the total navigation potential field.

[0009] In a preferred embodiment, the observation confidence in the voxel attribute is updated using linear update logic: for voxels located within the camera's view frustum and not obscured by obstacles as detected by line-of-sight tracking, the updated observation confidence is calculated using a linear update formula. This formula sets the updated observation confidence equal to the observation confidence before the update plus the product of the unobserved probability and the single effective observation gain coefficient, where the unobserved probability is defined as one minus the observation confidence before the update.

[0010] Regarding the construction of the aerodynamic physical field, this method constructs a diagonal damping matrix in the wind coordinate system, which consists of the damping coefficients in the downwind and crosswind directions. Then, based on the local wind field vector, a rotation matrix from the wind coordinate system to the world coordinate system is constructed. This rotation matrix is ​​then used to calculate the diagonal damping matrix through similarity transformation matrix operations. The result is multiplied by a gain factor based on turbulence intensity to obtain the aerodynamic impedance tensor. This tensor-based modeling method can accurately describe the anisotropic wind resistance characteristics.

[0011] Regarding the calculation of the information field, the calculation of the original viewpoint information entropy involves calculating the independent information entropy of each visible voxel at the viewpoint location using the Shannon entropy formula, and calculating the distance weight between the viewpoint location and the center of each visible voxel using the inverse square of the Euclidean distance. Then, the original viewpoint information entropy density at the viewpoint location is calculated using a weighted summation formula. Based on this, the process of generating the anisotropic information entropy convolution kernel includes: calculating eigenvalues ​​and eigenvectors from the aerodynamic impedance tensor using an eigenvalue decomposition algorithm; calculating the fuzzy covariance matrix using matrix multiplication and summation, making the value of the fuzzy covariance matrix equal to the sum of the products of each eigenvalue, the corresponding eigenvector outer product matrix, and the photoelectric conversion ratio coefficient; and constructing a three-dimensional Gaussian probability density function based on the fuzzy covariance matrix as the anisotropic information entropy convolution kernel. Through this mechanism, the spatial shape of the convolution kernel dynamically deforms according to the aerodynamic damping characteristics, realizing the anisotropic modulation of the information field by the physical field.

[0012] To generate the final effective information potential field, this method determines the integral spatial domain around the candidate viewpoint location and constructs a spatial decay operator. The spatial decay operator is defined as the product of the fuzzy penalty coefficient and the value of the anisotropic information entropy convolution kernel at the corresponding relative position. The product of the original viewpoint information entropy density and the spatial decay operator is calculated using an integral formula within the integral spatial domain to obtain the effective information potential value at that position.

[0013] In multi-field coupled decision-making, the input state of the reinforcement learning policy model at least includes normalized electrical characteristics, the mean aerodynamic potential energy of the local region, the sum of the entropy of the remaining unobserved information of the entire map, and the abrupt changes in the wind field. The weighted synthesis process involves calculating the addition of the static obstacle field and the weighted aerodynamic potential energy using a linear superposition formula, and then subtracting the weighted effective information potential energy field. The weights of each potential energy are determined by the coupling coefficients output by the reinforcement learning policy model, thereby achieving a dynamic balance between safety and efficiency at different task stages.

[0014] At the execution control level, the mechanism for dynamically adjusting the prediction step size includes: calculating the gradient magnitude of the aerodynamic potential energy at the current position through gradient magnitude calculation; calculating an exponential decay term through exponential function mapping, where the natural constant is used as the base and the negative value of the gradient magnitude is used as the exponent; and calculating the prediction step size for the current control cycle through numerical rounding logic, i.e., multiplying the product of the exponential decay term and the prediction time-domain adjustment factor by one, multiplying it by the basic prediction step size, and rounding down. Furthermore, the generation of velocity control commands includes velocity clamping logic: setting the normalized vector of the negative gradient direction of the total navigation potential field as the direction of motion, and calculating the dynamic velocity upper limit through an inverse proportional function relationship, making the dynamic velocity upper limit inversely proportional to the gradient magnitude of the aerodynamic potential energy at the current position, and finally selecting the smaller value between the maximum permissible flight speed and the dynamic velocity upper limit as the magnitude of the command speed.

[0015] A second aspect of this invention provides a dynamic path planning system for UAV inspection based on reinforcement learning. The system includes: an environment discretization modeling module for constructing a three-dimensional voxel mesh and maintaining voxel properties to provide basic environmental data for subsequent entropy calculations; a dynamic aerodynamic impedance field construction module for solving local wind fields and constructing aerodynamic impedance tensors, as well as calculating aerodynamic potential energy through quadratic form operations; and an effective viewpoint entropy field generation module for receiving the aerodynamic impedance tensor from the dynamic aerodynamic impedance field construction module to generate anisotropic information entropy convolution kernels and utilizing anisotropic information... The entropy convolution kernel calculates the effective information potential energy field by spatial convolution of the original viewpoint information entropy calculated based on the three-dimensional voxel mesh; the manifold parameter tuning module is used to output the coupling coefficient and prediction time-domain adjustment factor according to the state characteristics, and to obtain the total navigation potential field that integrates the aerodynamic potential energy and the effective information potential energy field through linear weighted synthesis calculation; and the path generation and execution module is used to calculate the gradient of the total navigation potential field through the gradient difference operator, and to dynamically adjust the prediction step size using the aerodynamic potential energy gradient magnitude from the dynamic aerodynamic impedance field construction module to generate speed control commands.

[0016] This invention provides a method and system for dynamic path planning of UAV inspection based on reinforcement learning. It has the following beneficial effects: 1. This invention constructs an aerodynamic impedance tensor characterizing the damping properties of spatial motion and uses an anisotropic convolution kernel defined by this tensor to spatially convolve and modulate the original viewpoint information entropy, thereby achieving anisotropic filtering of information gain by the aerodynamic environment. This mechanism transforms the physical risks of high turbulence or strong headwind regions into numerical suppression of information value, reducing the attractiveness of observed targets located in high damping regions to the navigation potential field. This guides UAVs to follow the airflow structure when planning their paths, naturally avoiding high-energy-consuming and high-risk airspaces, and improving flight safety and energy efficiency in complex wind field environments.

[0017] 2. This invention utilizes a pre-trained reinforcement learning strategy model to collect the state features of the UAV and the environment, and dynamically outputs coupling coefficients to weighted synthesize aerodynamic potential energy and effective information potential energy fields, realizing the adaptive fusion of multiple physical fields and information fields. This method overcomes the limitation of fixed weight parameters in the traditional potential field method, enabling the UAV to intelligently adjust its strategy center of gravity according to real-time states such as remaining power and the degree of wind field change. For example, in low power or high-risk states, priority is given to ensuring aerodynamic safety, while in stable states, information acquisition efficiency is maximized, thereby ensuring the robustness of the mission throughout its entire lifecycle.

[0018] 3. This invention establishes a dynamic mapping relationship between path planning granularity and environmental complexity by calculating the gradient modulus of aerodynamic potential energy and dynamically adjusting the prediction step size and speed command upper limit by combining the prediction time-domain adjustment factor. In areas with severe airflow disturbances or dense obstacles, the system automatically reduces the prediction step size and limits the flight speed to ensure control accuracy and stability; while in stable airspace, the step size is increased to improve the inspection speed, effectively avoiding path oscillation or overshoot caused by drastic changes in environmental gradient, and achieving a balance between control stability and inspection efficiency. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the system modules of the present invention.

[0020] Among them, 10 is the environmental discretization modeling module; 20 is the dynamic aerodynamic impedance field construction module; 30 is the effective viewpoint entropy field generation module; 40 is the manifold parameter tuning module; and 50 is the path generation and execution module. Detailed Implementation

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Reference Figure 2 This invention provides a dynamic planning system for UAV inspection paths based on reinforcement learning. The system includes a UAV platform, airborne sensing components, an airborne computing unit, and a flight control unit.

[0023] The drone platform, serving as the system's carrier, includes a frame, a power motor, propellers, and an electronic speed controller. The power motor is connected to the flight control unit via the electronic speed controller, receiving and executing flight control commands by changing its speed. The flight control unit receives lower-level control signals and outputs motor pulse-width modulation signals to maintain the drone's attitude stability.

[0024] The airborne sensing components are physically mounted on the drone platform to collect environmental data and the drone's own status data in real time. These components include an airspeed tube sensor, an inertial measurement unit, a lidar sensor, and a gimbal camera.

[0025] The pitot tube sensor is positioned at the nose of the drone to collect headwind air pressure data and transmit this pressure data to the onboard computing unit to calculate the relative air velocity.

[0026] The inertial measurement unit includes a three-axis accelerometer and a three-axis gyroscope, used to collect three-axis acceleration data and three-axis angular velocity data of the UAV.

[0027] LiDAR sensors are used to emit laser beams into the surrounding environment and receive the echoes to generate environmental point cloud data, which contains the spatial geometric coordinates of obstacles.

[0028] The gimbal camera includes a three-axis stabilized gimbal and an imaging sensor, used to acquire image data of the inspected target. The gimbal camera can output the gimbal's pitch, yaw, and roll angle data in real time.

[0029] The airborne computing unit communicates with the airborne sensing components and flight control unit via a data bus. The airborne computing unit includes a processor and a memory. The memory stores computer program instructions, and the processor executes these instructions to implement dynamic path planning logic. Logically, the airborne computing unit is divided into the following functional modules: environmental discretization modeling module 10, dynamic aerodynamic impedance field construction module 20, effective viewpoint entropy field generation module 30, manifold parameter tuning module 40, and path generation and execution module 50.

[0030] The environmental discretization modeling module 10 receives environmental point cloud data generated by the lidar sensor and pose data from the gimbal camera. This module divides the three-dimensional space containing the inspection target into a three-dimensional voxel mesh and assigns an occupancy probability attribute and an observation confidence attribute to each voxel. The occupancy probability attribute represents the likelihood of an obstacle existing within the voxel space, while the observation confidence attribute represents the degree to which the target surface information corresponding to that voxel is effectively acquired.

[0031] The dynamic aerodynamic impedance field construction module 20 receives relative air velocity data collected by the pitot tube sensor and acceleration residual data collected by the inertial measurement unit. Based on the aerodynamic parameters of the UAV, this module 20 calculates the local wind field vector and turbulence intensity in real time. Furthermore, it constructs aerodynamic impedance tensors at various locations in space. These tensors are symmetric matrices that characterize the energy consumption and damping properties of motion in different spatial directions.

[0032] The effective viewpoint entropy field generation module 30 is connected to the environment discretization modeling module 10 and the dynamic aerodynamic impedance field construction module 20. This module calculates the original information entropy of voxels within the field of view and receives the aerodynamic impedance tensor from the dynamic aerodynamic impedance field construction module 20. The module generates an anisotropic information entropy convolution kernel using the aerodynamic impedance tensor, and then performs spatial convolution on the original information entropy using this kernel to generate an effective information potential energy field. The value of this effective information potential energy field reflects the effective information gain that can be provided at various spatial locations, considering imaging blur caused by aerodynamic disturbances.

[0033] The manifold parameter tuning module 40 receives the UAV's current remaining battery power data, the average aerodynamic potential energy data output by the dynamic aerodynamic impedance field construction module 20, and the sum of remaining unobserved information entropy data output by the environment discretization modeling module 10. This manifold parameter tuning module 40 runs a pre-trained deep neural network policy model and outputs the aerodynamic potential field coupling coefficient, the information potential field coupling coefficient, and the prediction time-domain adjustment factor.

[0034] The path generation and execution module 50 receives the field data and parameters generated by the aforementioned modules. Based on the aerodynamic potential field coupling coefficient and the information potential field coupling coefficient, this module 50 linearly weights and synthesizes the aerodynamic impedance field and the effective information potential energy field to obtain the total navigation potential field. The module 50 then calculates the gradient vector of the total navigation potential field at the UAV's current position. Simultaneously, based on the prediction time-domain adjustment factor and the gradient magnitude of the aerodynamic impedance field, the module 50 determines the prediction step size for local path planning. Finally, the module 50 generates a velocity command along the negative gradient direction of the total navigation potential field and sends this velocity command to the flight control unit.

[0035] Reference Figure 1 The present invention also provides a dynamic planning method for UAV inspection paths based on reinforcement learning. The environment discretization modeling module 10 executes the method to transform the continuous inspection scenario in the physical world into a discrete numerical model that can be processed by a computer.

[0036] The environmental discretization modeling module 10 first establishes a global coordinate system, with the starting point of the inspection task or the location of the base station as the origin. Based on the pre-defined boundaries of the inspection area, this module constructs a three-dimensional cuboid bounding box that surrounds the target to be inspected (such as power poles or substation equipment).

[0037] The environment discretization modeling module 10 divides the aforementioned three-dimensional cuboid bounding box into a uniform mesh in space, generating a set of voxels composed of multiple cubic units. Each cubic unit is called a voxel. ,in This is a unique index for the voxel. The resolution of the voxel's side length is set to [value]. The resolution is set according to the required inspection accuracy.

[0038] The system stores each voxel in computer memory. Allocate dedicated storage space to maintain two core state variables: occupancy probability. and observation confidence .

[0039] Regarding the probability of occupancy The environment discretization modeling module 10 receives the raw point cloud data output by the lidar sensor. This module uses a Bayesian filtering algorithm to recursively estimate the occupancy state of the environment. To avoid numerical instability caused by probability multiplication, this module uses logarithmic probability for storage and updating. Specifically, for any given time... When the laser beam of the lidar hits a voxel When the laser beam passes through a voxel, the system increases the logarithmic probability value of that voxel; When reaching more distant obstacles, the system reduces the number of voxels. The logarithmic odds value. The probability of occupancy. With logarithmic odds The conversion relationships are as follows: ; in, This represents an exponential function. The system sets a threshold, when... When the value exceeds this threshold, the voxel is marked as an obstacle voxel and treated as an impassable area in subsequent path planning.

[0040] Regarding observation confidence In the update, the environment discretization modeling module 10 receives real-time pose data (position and quaternion pose) from the gimbal camera and the camera's intrinsic parameter matrix (focal length, principal point coordinates). Based on the camera's horizontal and vertical field of view angles, this module constructs the current-moment frustum geometric model in three-dimensional space.

[0041] The environment discretization modeling module 10 iterates through all voxels located within the aforementioned view frustum geometry. For any voxel within the view frustum... The module performs line-of-sight occlusion detection. In specific implementation, the module moves from the camera's optical center to the voxel. A virtual ray is emitted from the center of the ray; if the virtual ray reaches... If a voxel has not previously intersected with any voxels marked as obstacles, then that voxel is considered an obstacle. Currently visible.

[0042] For a voxel determined to be currently visible, the system determines its observation confidence level. Incremental updates are performed. This update process simulates the cumulative effect of information acquisition, meaning that the uncertainty regarding the surface information of the area gradually decreases as the number of observations increases. The update of the observation confidence follows these rules: ; in, The updated observation confidence level, The confidence level of the observations before the update. This is the gain coefficient for a single effective observation, and its value ranges from 0 to 1.

[0043] Initially, the observation confidence of all voxels It was set to 0. As the inspection process progressed, the voxels covered by the camera... The value gradually approaches 1. When When the preset saturation threshold is reached, the system determines that the target area represented by the voxel has been inspected and covered.

[0044] Through the above steps, the environmental discretization modeling module 10 maintains and outputs two three-dimensional numerical fields in real time: one is an occupancy grid map representing physical geometric constraints, and the other is an information overlay map representing the task completion progress. These two map data are transmitted to subsequent modules in real time as the basic input data for generating the dynamic aerodynamic impedance field and the effective viewpoint entropy field.

[0045] The Dynamic Aerodynamic Impedance Field Construction Module 20 executes this process, the core purpose of which is to transform the invisible airflow environment into a mathematical scalar field that can be used for path planning.

[0046] The dynamic aerodynamic impedance field construction module 20 first performs real-time calculation of local wind field parameters. This module simultaneously acquires vacuum speed data output from the pitot tube sensor and ground speed data output from the flight control unit. Through vector subtraction, the module calculates the current local wind field vector. The wind field vector It includes information on wind speed and direction.

[0047] To quantify the instability of the airflow, this module maintains a length of A time sliding window is used. Within this time window, the module calculates the standard deviation of the wind field vector magnitude, and defines this standard deviation as the turbulence intensity. The higher the turbulence intensity value, the more severe the airflow disturbance in the current region.

[0048] Subsequently, the dynamic aerodynamic impedance field construction module 20 constructs the aerodynamic impedance tensor based on the aforementioned wind field parameters. This tensor is a 3×3 symmetric positive definite matrix used to describe the motion damping characteristics of a point in space in different directions. The construction process includes the following specific technical actions: First, construct the diagonal damping matrix in the wind coordinate system. The matrix is ​​defined as follows: ; in, Diagonal operator, The damping coefficient is the damping coefficient in the downwind / upwind direction. This represents the damping coefficient in the crosswind direction. Based on the asymmetric aerodynamic characteristics of rotary-wing UAVs, it is typically set as follows: This is to demonstrate the greater impact of crosswinds on flight stability.

[0049] Second, calculate the rotation matrix. This module is based on wind field vectors. Construct the rotation matrix from the wind coordinate system to the world coordinate system using the direction cosine in the world coordinate system. .

[0050] Third, synthesize the aerodynamic impedance tensor. The module maps the diagonal damping matrix back to the world coordinate system and introduces turbulence intensity. As an overall gain factor: ; in, The preset turbulence sensitivity coefficient, Rotation matrix The transpose of .

[0051] Finally, the dynamic aerodynamic impedance field construction module 20 calculates the spatial position. aerodynamic potential energy at the location This module uses aerodynamic impedance to perform a quadratic operation on the normalized wind speed vector to obtain scalar potential energy: ; in, Based on the value of the basic environment, Wind field vector The transpose of .

[0052] The aerodynamic potential energy value The data is mapped to the same voxel mesh structure as the environment discretization modeling module 10. For each voxel, a higher aerodynamic potential energy value indicates greater energy consumption and lower stability during flight in that region. This data field serves as the core input for adjusting the effectiveness of information entropy in subsequent steps.

[0053] The effective viewpoint entropy field generation module 30 performs this calculation step, which aims to quantify the theoretical information gain for the inspection target that the UAV can acquire when it is in a specific spatial position and attitude under ideal static conditions.

[0054] The effective viewpoint entropy field generation module 30 first iterates through the voxel set output by the environment discretization modeling module 10. For each voxel in the set... This module is based on its current observation confidence level. Calculate the independent information entropy value of this voxel. This calculation, based on Shannon's information theory, is used to measure the uncertainty of voxel states. When the observation confidence level... When the entropy is 0.5 (i.e., in a completely unknown state), the voxel has the maximum entropy value; when When the entropy approaches 0 or 1 (i.e., the state is determined), the entropy value approaches 0. The calculation formula is as follows: ; in, Represents the natural logarithm operation. Observation confidence level.

[0055] Subsequently, the module determines the candidate viewpoint positions in three-dimensional space. An evaluation is performed to calculate the original viewpoint information entropy density at that location. The calculation process includes two sub-steps: field-of-view clipping and occlusion removal.

[0056] In the field-of-view clipping step, the module constructs a virtual view frustum based on the intrinsic parameter matrix of the gimbal camera. This view frustum is bounded by the near clipping plane, the far clipping plane, and the field-of-view boundary plane. The module then selects a subset of candidate voxels whose geometric centers are located within this view frustum.

[0057] In the occlusion culling step, the module performs gaze-tracking (RayCasting) detection on the aforementioned candidate voxel subset. The effective viewpoint entropy field generation module 30 generates the effective viewpoint entropy field from the viewpoint position. To each candidate voxel A virtual ray is emitted from the center. If the ray reaches... Previously occupied with other probabilities If an obstacle object with a value higher than a preset threshold intersects with another object, then the object is considered a voxel. If an element is obscured, its information gain is not included in the current viewpoint's calculation.

[0058] For a voxel that passes the visibility test, the module calculates its relationship to the viewpoint. The information contribution. To reflect the optical characteristic of imaging resolution decreasing with distance, the module introduces a distance weighting factor. Original viewpoint information entropy density. Defined as the weighted sum of the independent information entropies of all visible voxels: ; in, Indicates the location A collection of voxels visible in the space. This represents the Euclidean distance between the viewpoint position and the voxel center. This formula shows that a viewpoint closer to the target can obtain a higher original information entropy density value under the same field of view coverage. The calculation result generates a three-dimensional scalar field based solely on geometric positional relationships, serving as the basis data for subsequent convolutional modulation incorporating aerodynamic properties.

[0059] The effective viewpoint entropy field generation module 30 performs this generation step, and its core logic lies in constructing a spatial transfer function that can dynamically reflect the impact of aerodynamic disturbances on imaging quality.

[0060] The effective viewpoint entropy field generation module 30 first receives the current position output by the dynamic aerodynamic impedance field construction module 20. aerodynamic impedance tensor at point This aerodynamic impedance tensor, as a second-order symmetric tensor, contains the intensity distribution and directional characteristics of the airflow disturbance at the current spatial location.

[0061] To map abstract aerodynamic data to a blurring effect in image space, this module maps the aerodynamic impedance tensor. Perform eigenvalue decomposition. This operation solves for the three eigenvalues ​​of the tensor. and the corresponding three orthogonal eigenvectors The magnitude of the eigenvalue directly represents the aerodynamic disturbance energy in the direction of the corresponding eigenvector. For example, if... If the value is greater than the other two characteristic values, it indicates that the UAV mainly bears the load along the direction of travel at this location. The intense aerodynamic load in that direction will cause the airborne camera to produce high-frequency vibration displacement along that direction.

[0062] Based on the above eigenvalue decomposition results, this module constructs a fuzzy covariance matrix. The covariance matrix is ​​used to define the geometry of the 3D Gaussian convolution kernel. During construction, the module incorporates a photoelectric conversion scaling factor from an imaging system. This maps aerodynamic eigenvalues ​​to the variance of a Gaussian distribution. The formula for constructing the fuzzy covariance matrix is ​​as follows: ; in, This indicates transpose.

[0063] This step ensures that the principal axis of the fuzzy covariance matrix is ​​aligned with the principal axis of the aerodynamic impedance tensor, thus achieving the physical inheritance of the perturbation directionality.

[0064] Subsequently, the effective viewpoint entropy field generation module 30 uses the aforementioned fuzzy covariance matrix to instantiate anisotropic information entropy convolution kernels. The convolution kernel is a probability density function defined in three-dimensional space, describing the spatial dispersion distribution of unit information under aerodynamic disturbances. The analytical expression of the convolution kernel is defined as follows: ; in, This represents the relative displacement vector from the center of the convolution kernel to neighboring points. Represents matrix determinant operations. The inverse of the fuzzy covariance matrix.

[0065] Through the above steps, the system generates a mathematical kernel function that dynamically deforms with spatial location. In regions of stable airflow, The eigenvalues ​​are small. The corresponding ellipsoidal volume shrinks, and the convolution kernel approaches the Dirac principle. The function implies clear imaging and no information diffusion; in regions of strong turbulence, As the eigenvalues ​​increase, The corresponding ellipsoidal volume expands along the perturbation direction, and the convolution kernel exhibits a wide lobe shape, indicating severe spatial smearing and aliasing of information, resulting in a decrease in effective resolution. This convolution kernel will be used in subsequent steps to perform spatial filtering on the original viewpoint information entropy.

[0066] The effective viewpoint entropy field generation module 30 performs this step, transforming the aerodynamic instability of the environment into a penalty for the amount of information acquired through mathematical convolution modulation operations, thereby generating the potential energy field ultimately used for navigation.

[0067] The effective viewpoint entropy field generation module 30 first determines the spatial integration domain of the convolution operation. This integration domain is typically set to the current candidate viewpoint position of the UAV. Centered on a local three-dimensional spatial region covering its sensor's field of view, the module extracts the raw viewpoint information entropy density of all voxels within this region. As the integrand, where Let be the integration variable, representing the spatial coordinates within the integration domain.

[0068] The effective viewpoint entropy field generation module 30 calls the anisotropic information entropy convolution kernel generated in the aforementioned steps. At this point, the convolution kernel is considered a spatial filter, the shape of which is determined by the drone's position. The aerodynamic impedance tensor at a given point is uniquely determined.

[0069] The effective viewpoint entropy field generation module 30 performs weighted convolution integral operations to calculate the position. Effective information potential value at the location This operation is not a simple smoothing filter, but a utility calculation based on the principle of fuzzy loss. Specifically, the module uses the following mathematical model to describe the process of retaining effective information: ; in, The preset fuzziness penalty coefficient is used to adjust the degree to which aerodynamic fuzziness reduces the information value. (The terms in parentheses are...) This constitutes a spatial decay operator. Integration region.

[0070] During this operation, the system evaluates the erosion effect of the aerodynamic convolution kernel on the original information point by point. If the UAV is located in a region of stable airflow, the convolution kernel... It exhibits a highly convergent pulse shape in space, and its value is only... Extremely close The time term is non-zero, making the decay term in the integral result extremely small. It highly preserves the original information entropy The value of represents a high potential energy characteristic (which manifests as a deep valley in the gravitational field model, exerting a strong attraction on the drone).

[0071] Conversely, if the drone is located in a region of strong turbulence or significant wind shear, the convolution kernel... Significant spatial dispersion occurs along the direction of aerodynamic disturbance. At this point, the attenuation operator has a significant effect over a large spatial range. Weighted subtraction is performed. This means that even if there are a large number of high-entropy voxels within the field of view, the theoretical information gain provided by these voxels will be significantly offset due to the wide lobe coverage of the convolution kernel.

[0072] Finally, the module will calculate the result. Mapping back to a discrete grid, a continuous effective information potential energy field is formed. Geometrically, this effective information potential energy field represents a series of gravitational traps. However, unlike traditional methods, the depth of these gravitational traps depends not only on the amount of target information but also on local aerodynamic stability. Regions that were originally information-rich but had dangerous airflow are now filled with aerodynamic convolution kernels, no longer effectively attracting the UAV. This achieves a physical entanglement of detection efficiency and flight stability at the source level of path planning.

[0073] The reinforcement learning-driven manifold parameter tuning module 40 performs this step. Its core function is to use the nonlinear fitting capability of deep neural networks to dynamically adjust the topological coupling relationship between the physical field and the information field according to the current task state.

[0074] The manifold parameter tuning module 40 first constructs the current state observation vector of the system. This module obtains battery management system status data from the flight control unit via the data bus, calculates the ratio of the current remaining battery power to the total battery capacity, and generates a normalized battery power characteristic. The manifold parameter tuning module 40 obtains the average aerodynamic potential energy in a local region centered on the current position of the UAV from the dynamic aerodynamic impedance field construction module 20. This serves as a characteristic of the severity of the environment. Simultaneously, this module obtains the sum of the original information entropy of the remaining unobserved voxels in the entire image from the environment discretization modeling module 10. This serves as a feature of the remaining workload of the task. Furthermore, this module calculates the magnitude of the wind field vector difference between the previous control cycle and the current control cycle, generating wind field abrupt change features. The four feature components mentioned above are concatenated and combined to form the state observation vector. .

[0075] The manifold parameter tuning module 40 will use the state observation vector The input is fed into a pre-trained deep neural network policy model. This policy model employs a multilayer perceptron (MLP) structure, comprising an input layer, several hidden layers, and an output layer. The number of neurons in the input layer is consistent with the dimension of the state observation vector. The hidden layers use modified linear units as activation functions to extract high-dimensional nonlinear mappings of state features. The output layer uses hyperbolic tangent or sigmoid activation functions to constrain the network output within a preset numerical range.

[0076] The forward propagation calculation of this strategy model outputs the action vector at the current moment. The action vector contains three scalar parameters: the aerodynamic potential coupling coefficient. Information potential field coupling coefficient And predictive time-domain adjustment factors Aerodynamic potential coupling coefficient The repulsion weight of aerodynamic impedance to the overall navigation field, and the information potential coupling coefficient. Used to adjust the attraction weight of effective information gain to the total navigation field, predicting the time-domain adjustment factor. Used for scaling the time step in subsequent path planning steps.

[0077] To ensure that the reinforcement learning policy model can accurately output the coupling coefficient and predict the temporal adjustment factor in practical applications, this embodiment uses a combination of offline training and online fine-tuning to construct the model. The specific training environment and mechanism are as follows: Training is conducted on a physics-based high-fidelity simulation platform (such as AirSim or Gazebo). To simulate realistic aerodynamic environments, a simplified computational fluid dynamics (CFD) solver or a Perlin noise field generator is integrated into the environment to generate dynamic wind fields with different flow velocities, directions, and turbulence intensities in three-dimensional space. Simultaneously, obstacles of varying densities (such as buildings and trees) are randomly generated within the simulation environment to construct diverse training scenarios and enhance the model's generalization ability to environmental changes.

[0078] The training objective of the model is to maximize the cumulative expected return. The reward function designed in this embodiment... It consists of three parts: Information gain reward ( A positive reward is given when the drone's movement causes the field of view to cover new unknown voxels or significantly reduces the uncertainty of the entire map. The reward value is proportional to the total amount of information entropy eliminated within the current step.

[0079] Pneumatic safety penalty ( When a drone enters an area with high aerodynamic potential energy (i.e., strong wind resistance or turbulence), a negative penalty is imposed based on the modulus of the aerodynamic impedance tensor; if a collision occurs or the drone exceeds the flight envelope, a larger termination penalty is imposed.

[0080] Energy consumption and efficiency penalty ( To expedite mission completion and conserve power, a small survival penalty is imposed after each control cycle, along with an additional energy penalty based on the product of current airspeed and wind resistance (simulated power consumption). The final single-step reward function can be expressed as a linear weighted sum of the above three factors. .

[0081] The proximal policy optimization algorithm is adopted as the benchmark algorithm because it has good convergence and stability in continuous action space control tasks.

[0082] State space: corresponding to the normalized charge, local aerodynamic potential energy mean, and global residual entropy mentioned in the claims.

[0083] Action space: Outputs a continuous two-dimensional vector, corresponding to the physical field information field coupling coefficients ( [0,1]) and prediction time-domain adjustment factor ( ).

[0084] The training data is entirely derived from the self-play and trial-and-error process within the aforementioned simulation environment. Through millions of time-step iterations, the model learns how to dynamically balance where to look (information gains) and how to fly safely (aerodynamic costs) under different remaining battery power and wind conditions. After training, the policy network parameters are extracted and deployed to the airborne computing platform.

[0085] After obtaining the aforementioned coupling coefficients, the manifold parameter tuning module 40 performs the synthesis operation of the composite potential energy manifold. This module superimposes the static obstacle field, the dynamic aerodynamic impedance field, and the effective viewpoint entropy field in three-dimensional space. Among these, the static obstacle field... It is a standard repulsion field generated based on Euclidean distance transformation, used to ensure basic geometric obstacle avoidance safety. Total navigation potential field. The synthesis formula is defined as: ; in, and By taking a positive value, a high potential energy barrier is constructed, enabling the drone to stay away from obstacles and areas of strong airflow during gradient descent; The negative sign at the beginning transforms it into a low potential energy trough, propelling the drone towards the high information gain region after aerodynamic correction. Aerodynamic or spatial constraints, Information features.

[0086] Through the above process, this module integrates the originally independent physical constraints and information requirements into a unified scalar field on the Riemannian manifold space. The topology of this scalar field evolves dynamically over time: when the power is sufficient and the wind field is stable, the policy network increases. This deepens the information gravitational trap in the potential field, guiding the drone to actively explore; when encountering sudden gusts of wind or low battery, the strategy network increases... This raises the potential energy level of the danger zone, forcing drones to prioritize choosing a smooth path to return home or avoid danger.

[0087] The path generation and execution module 50 performs this step, transforming the abstract mathematical potential field generated in the previous steps into specific control commands at the UAV's underlying level, and realizing adaptive adjustment of the control cycle.

[0088] The path generation and execution module 50 first obtains the current time of the UAV. Three-dimensional spatial coordinates This module synthesizes the total navigation potential field by the manifold parameter tuning module 40. Perform spatial gradient calculations. Since the total navigation potential is a scalar field defined on a discrete voxel grid, this module uses the central difference method or the Sobel operator to approximate the gradient vector at the current position. .

[0089] gradient vector The three components correspond to The rate of change of potential energy along the axis. This gradient vector indicates the direction of the fastest increase in potential energy at the current position. To guide the UAV towards the point of lowest potential energy (i.e., the region of maximum information gain and minimum aerodynamic cost), this module determines the desired motion direction vector. The normalized vector in the opposite direction of the gradient vector: ; in, For Hamiltonian operators, This represents the Euclidean norm.

[0090] Meanwhile, the path generation and execution module 50 performs elastic adjustment calculations in the prediction time domain. This step aims to resolve the contradiction that a fixed control frequency cannot simultaneously adapt to stable flight and severe turbulence. This module reads the prediction time domain adjustment factor output by the manifold parameter tuning module 40. and the gradient magnitude of the aerodynamic impedance potential energy field at the current position. This module dynamically calculates the prediction step size of the current control cycle based on the following mapping relationship. : ; in, The preset base prediction step size for the system, This represents the integer operation. This represents an exponential function. The technical advantage of this formula is that when the UAV is in a region with drastic changes in aerodynamic potential energy gradient (i.e., a complex and variable airflow environment), the value of the exponential term decreases, thus reducing the prediction step size. When the flow is shortened, the system enters a short-sighted, high-frequency response mode to quickly respond to local airflow disturbances; conversely, in areas of stable airflow, the prediction step size is shortened. Extend the system's trajectory to improve inspection efficiency.

[0091] Based on the determined motion direction vector With prediction step size This module generates the final speed control command. To prevent excessive speed commands from being generated in steep potential fields, which could lead to actuator saturation, the module introduces speed clamping logic: ; in, The maximum permissible flight speed for drones, The velocity gain constant is... To prevent the denominator from being zero, this logic ensures that in harsh aerodynamic environments (where the denominator is large), the UAV automatically reduces its flight speed to maintain attitude control stability.

[0092] The path generation and execution module 50 will generate The data is transmitted to the flight control unit via the data bus, driving the motors to perform maneuvers. After the UAV moves, the onboard sensing components collect new environmental and status data, and the system returns to the environmental discretization modeling module 10 to enter the next control cycle, until the total entropy of the remaining unobserved information in the entire map is lower than the preset mission termination threshold.

Claims

1. A dynamic path planning method for UAV inspection based on reinforcement learning, characterized in that, Includes the following steps: A three-dimensional voxel mesh is constructed using data collected by airborne sensing components and voxel properties are updated in real time to provide environmental basis data for viewpoint entropy calculation. An aerodynamic impedance tensor characterizing the damping properties of space motion is constructed based on airspeed and inertia data, and aerodynamic potential energy is calculated using the aerodynamic impedance tensor through quadratic form operations. The original viewpoint information entropy of the voxels in the three-dimensional voxel grid is calculated by field-of-view geometric analysis, and the original viewpoint information entropy is spatially convolved by the anisotropic information entropy convolution kernel defined by the aerodynamic impedance tensor characteristics to generate an effective information potential energy field after aerodynamic fuzz modulation. The UAV and environmental state features are collected and input into the pre-trained reinforcement learning strategy model. The coupling coefficient and prediction time-domain adjustment factor are obtained. The aerodynamic potential energy and the effective information potential energy field are linearly weighted and synthesized according to the coupling coefficient to obtain the total navigation potential field. The gradient vector of the total navigation potential field is calculated by the gradient difference operator, and the prediction step size is dynamically adjusted by combining the gradient magnitude of the aerodynamic potential energy with the prediction time-domain adjustment factor to generate a velocity control command along the negative gradient direction of the total navigation potential field.

2. The method for dynamic planning of UAV inspection paths based on reinforcement learning according to claim 1, characterized in that, The step of updating the observation confidence in the voxel attribute includes: For voxels located within the camera's view frustum and not obstructed by obstacles as detected by line-of-sight tracking, the updated observation confidence is calculated using a linear update formula. This formula sets the updated observation confidence to the product of the original observation confidence plus the unobserved probability and the single effective observation gain coefficient. The unobserved probability is one minus the original observation confidence.

3. The method for dynamic planning of UAV inspection paths based on reinforcement learning according to claim 1, characterized in that, The steps for constructing the aerodynamic impedance tensor include: Construct a diagonal damping matrix in the wind coordinate system, which consists of the damping coefficient in the downwind direction and the damping coefficient in the crosswind direction; construct a rotation matrix from the wind coordinate system to the world coordinate system based on the local wind field vector; The diagonal damping matrix is ​​calculated using the rotation matrix through similarity transformation matrix operations, and the result is multiplied by a gain factor based on turbulence intensity to obtain the aerodynamic impedance tensor.

4. The method for dynamic planning of UAV inspection paths based on reinforcement learning according to claim 1, characterized in that, The calculation steps for the original viewpoint information entropy include: The independent information entropy of each voxel visible at the viewpoint is calculated using the Shannon entropy formula; the distance weight between the viewpoint and the center of each visible voxel is calculated using the inverse square of the Euclidean distance. The original viewpoint information entropy density of the viewpoint position is calculated using a weighted summation formula, which accumulates the product of the independent information entropy of all visible voxels with their corresponding distance weights.

5. The method for dynamic planning of UAV inspection paths based on reinforcement learning according to claim 1, characterized in that, The steps for generating the anisotropic information entropy convolution kernel include: The aerodynamic impedance tensor is calculated using an eigenvalue decomposition algorithm to obtain eigenvalues ​​and eigenvectors; The fuzzy covariance matrix is ​​calculated by matrix multiplication and summation, and the value of the fuzzy covariance matrix is ​​equal to the sum of the products of each eigenvalue, the corresponding eigenvector outer product matrix, and the photoelectric conversion ratio coefficient. Based on the fuzzy covariance matrix, a three-dimensional Gaussian probability density function is constructed as the anisotropic information entropy convolution kernel, so that the spatial shape of the anisotropic information entropy convolution kernel dynamically deforms with the aerodynamic damping characteristics.

6. The method for dynamic planning of UAV inspection paths based on reinforcement learning according to claim 1, characterized in that, The steps for generating the effective information potential field include: Determine the integral space domain around the candidate viewpoint position; construct a spatial decay operator, and calculate it by numerical subtraction operation. The spatial decay operator is defined as the product of the fuzzy penalty coefficient and the value of the anisotropic information entropy convolution kernel at the corresponding relative position. The effective information potential value of the candidate viewpoint position is obtained by calculating the product of the original viewpoint information entropy density and the spatial attenuation operator in the integral space domain using an integral formula.

7. The method for dynamic planning of UAV inspection paths based on reinforcement learning according to claim 1, characterized in that, The steps to obtain the total navigation potential field include: The input state of the reinforcement learning strategy model includes at least the normalized electrical charge features, the mean aerodynamic potential energy of the local region, the sum of the entropy of the remaining unobserved information in the whole map, and the wind field abrupt change features. The weighted synthesis process involves calculating by adding the static obstacle field to the weighted aerodynamic potential energy using a linear superposition formula, and then subtracting the weighted effective information potential energy field; wherein the weight of each potential energy is determined by the coupling coefficient output by the reinforcement learning policy model.

8. The method for dynamic planning of UAV inspection paths based on reinforcement learning according to claim 1, characterized in that, The steps for dynamically adjusting the prediction step size include: The gradient magnitude of the aerodynamic potential energy at the current position is calculated by gradient magnitude calculation; the exponential decay term is calculated by exponential function mapping, wherein the exponential decay term is an exponential function of the natural constant and its exponent is the negative value of the gradient magnitude. The prediction step size of the current control cycle is calculated by numerical rounding logic. The numerical rounding logic adds one to the product of the exponential decay term and the prediction time-domain adjustment factor, multiplies it by the basic prediction step size, and then rounds it down.

9. The method for dynamic planning of UAV inspection paths based on reinforcement learning according to claim 1, characterized in that, The steps for generating speed control commands include speed clamping logic: The normalized vector of the negative gradient direction of the total navigation potential field is set as the direction of motion; The dynamic velocity upper limit is calculated using an inverse proportional function relationship, such that the dynamic velocity upper limit is inversely proportional to the gradient modulus of the aerodynamic potential energy at the current position. The smaller value between the maximum permissible flight speed and the dynamic speed limit is selected as the final command speed through numerical comparison logic.

10. A dynamic path planning system for UAV inspection based on reinforcement learning, characterized in that, The method for dynamic planning of UAV inspection paths based on reinforcement learning as described in any one of claims 1-9 includes: The environment discretization modeling module (10) is used to construct a three-dimensional voxel mesh and maintain voxel properties, providing environmental basic data for subsequent entropy calculation; The dynamic aerodynamic impedance field construction module (20) is used to solve the local wind field and construct the aerodynamic impedance tensor, as well as to calculate the aerodynamic potential energy through quadratic operations. The effective viewpoint entropy field generation module (30) is used to receive the aerodynamic impedance tensor from the dynamic aerodynamic impedance field construction module to generate an anisotropic information entropy convolution kernel, and use the anisotropic information entropy convolution kernel to calculate the original viewpoint information entropy calculated based on the three-dimensional voxel grid through spatial convolution to generate an effective information potential energy field. The manifold parameter tuning module (40) is used to output the coupling coefficient and predict the time-domain adjustment factor according to the state characteristics, and calculate the total navigation potential field that integrates aerodynamic potential energy and effective information potential energy field by linear weighted synthesis. The path generation and execution module (50) is used to calculate the total navigation potential field gradient by using the gradient difference operator, and dynamically adjust the prediction step size using the aerodynamic potential energy gradient modulus from the dynamic aerodynamic impedance field construction module to generate speed control commands.

Images