Bridge unmanned aerial vehicle autonomous inspection method based on gaussian scene perception and reinforcement learning

Abstract

The application proposes a bridge unmanned aerial vehicle autonomous inspection method based on Gaussian scene perception and reinforcement learning. The application realizes autonomous coverage inspection, damage target observation, path decision training and flight control closed loop verification of the unmanned aerial vehicle in the complex bridge structure environment by constructing a three-dimensional Gaussian training scene of the bridge, combining Gaussian new perspective rendering micro service, Euclidean symbol distance field safety constraint, multi-modal observation space, multi-target reward function, PPO model-free reinforcement learning method, SHAC differentiable policy optimization method and PX4 / Gazebo / MAVROS flight control closed loop feedback mechanism.

Description

Technical Field

[0001] This invention relates to the technical fields of structural health monitoring, autonomous UAV inspection, intelligent bridge structure detection, reinforcement learning, 3D Gaussian splash mapping, neural rendering, robot autonomous navigation, and UAV flight control, particularly to an autonomous UAV inspection method for bridges based on Gaussian scene perception and reinforcement learning. Specifically, it relates to an autonomous UAV inspection method for bridges that utilizes a 3D Gaussian scene to provide realistic visual observation, combined with geometric safety constraints, multimodal observation space, multi-objective reward function, differentiable strategy optimization, flight control closed-loop feedback, and reinforcement learning training. Background Technology

[0002] The purpose of UAV bridge inspection is to enable UAVs to autonomously select observation locations, plan safe flight paths, and complete high-quality inspection data collection in complex structural spaces. Areas such as the bottom of bridge beams, piers, supports, cable anchorage zones, and the interior of box girders often suffer from problems such as narrow spaces, structural obstructions, uneven lighting, airflow disturbances, and missing GNSS signals. Manually pre-setting flight paths is difficult to cover all critical components and also makes it difficult to adjust the observation angle in real time according to changes in the local environment.

[0003] Existing UAV inspection training methods mainly include manual flight path planning, traditional physical simulation reinforcement learning, and visual strategy learning based on real images. Manual flight path planning relies on engineers to pre-set waypoints. Although simple to implement, it is difficult to guarantee coverage integrity and observation angle quality when facing multi-obstructed areas such as the bottom of bridge beams, the sides of piers, and supports. At the same time, manual flight paths lack the ability to dynamically adjust inspection priorities based on local risks and defects.

[0004] Traditional physics or graphics simulation environments, such as Gazebo, AirSim, and Unreal Engine, can provide collision detection, dynamics simulation, and large-scale interactive capabilities. However, there is a significant domain difference between their visual observations and real bridge scenes, making it difficult to realistically depict concrete surface textures, steel box girder boundaries, localized crack and spalling areas, low-light environments under beams, and complex sunlight conditions. Using high-quality ray tracing rendering to improve realism would significantly reduce reinforcement learning sampling efficiency, making it difficult to meet the needs of large-scale policy optimization.

[0005] The 3D Gaussian splashing rendering method can reconstruct photorealistic scenes based on real images and point cloud data, and efficiently generate new perspective images on GPUs, providing a new technical path for reducing the differences in the simulation visual domain. However, 3D Gaussian scenes are usually only used as offline mapping results or new perspective synthesis models, and cannot yet be directly used as a reinforcement learning training environment for UAV autonomous inspection. To use them for policy training, further issues need to be addressed, such as decoupling Gaussian rendering from dynamic simulation, fast querying of geometric safety distance, construction of coverage rewards, semantic priority expression, policy optimization, and closed-loop feedback for realistic flight control.

[0006] Furthermore, existing reinforcement learning training frameworks typically employ idealized pose updates, where policy actions directly alter the agent's position or attitude, failing to adequately consider factors such as velocity saturation, inertial response, control latency, flight control attitude stability, and physical collision risks inherent in real-world drones. Consequently, actions learned in simulations are prone to execution deviations on real-world drone platforms.

[0007] To address the aforementioned shortcomings, this invention proposes an autonomous inspection method for UAVs based on Gaussian scene perception and reinforcement learning. This method integrates the new perspective rendering capability of 3D Gaussian scenes, Euclidean Signed Distance Field (ESDF) geometrical safety constraints, coverage sampling points, semantic priority rewards, Proximal Policy Optimization (PPO), and Short-Horizon Actor-Critic (SHAC) policy training methods, along with the PX4 flight control system, Gazebo physics simulation platform, and MAVROS communication interface, into a single training framework. This enables the training of autonomous inspection strategies for UAVs that combine realistic visual observation, computable geometrical constraints, and realistic dynamic feedback. Summary of the Invention

[0008] The purpose of this invention is to address the problems in existing technologies by proposing an autonomous bridge inspection method using unmanned aerial vehicles (UAVs) based on Gaussian scene perception and reinforcement learning. This method can be used for autonomous, full-coverage inspection of bridge beam bottoms, piers, supports, towers, cable anchorage zones, building corridors, industrial plants, tunnels, underground spaces, and other complex structural environments. It includes defect detection, inspection path planning, simulation-to-real-world transfer training, and flight control closed-loop verification.

[0009] This invention is achieved through the following technical solution: This invention proposes an autonomous bridge inspection method using unmanned aerial vehicles (UAVs) based on Gaussian scene perception and reinforcement learning, the method comprising: Step 1: Construct a 3D Gaussian training scene for UAV autonomous inspection to represent the texture, geometry, depth, and semantic information of the bridge structure; Step 2: Establish a Gaussian rendering and geometric safety constraint module to generate UAV visual observations in real time and complete safe distance queries, collision judgments, and coverage statistics; Step 3: Construct a reinforcement learning task model for UAV autonomous inspection, taking visual observation, task status, local geometric information and flight control status as inputs, and speed commands as action outputs; Step 4: Design a multi-objective reward function for coverage inspection to uniformly evaluate coverage gain, information gain, pose alignment, semantic priority, security risks, and invalid behaviors; Step 5: Train the UAV autonomous inspection policy network using the near-end policy optimization method and the short-time domain differentiable policy optimization method, and improve the policy robustness through the domain randomization mechanism; Step Six: Integrate the trained UAV autonomous inspection strategy into the flight control closed-loop system to achieve closed-loop verification, flight execution, and engineering deployment of the inspection strategy.

[0010] Furthermore, step one specifically includes: Step 11: Obtain multi-source scene data of the inspection object; Steps 1 and 2: Establish bridge scene pose constraints and initialize 3D Gaussian primitives. Use the precise camera pose output by the virtual camera in the Gazebo simulation environment as the pose input for training the 3D Gaussian scene. Then, initialize the Gaussian primitive positions by sampling point clouds from the bridge mesh model, reconstructing point clouds from multiple views, or back-projecting point clouds from the simulation depth. Step 13: Jointly optimize the 3D Gaussian scene parameters of the bridge. Utilize multi-view RGB image photometric reconstruction loss, structural similarity loss, simulation depth supervision term, geometric anchoring constraints, and depth correlation constraints to jointly optimize the Gaussian meta-parameters, so that the rendered image is consistent with the bridge simulation observation in terms of color, structure, depth, and geometric distribution. Step 14: Construct additional scenario information for bridge inspection tasks.

[0011] Furthermore, step two specifically includes: Step 21: Before reinforcement learning training begins, load the 3D Gaussian scene model into an independent rendering process, keep the Gaussian model resident in video memory, and provide on-demand rendering services to the outside world through a lightweight interface. Step 22: At each time step, the physics simulator or flight control system outputs the UAV's six-degree-of-freedom pose in the global coordinate system, based on the camera's extrinsic parameters. The drone system pose is converted into a camera pose, and the camera pose is then passed to the Gaussian rendering backend. Steps 2 and 3: The Gaussian rendering backend generates an RGB image, depth map, and semantic map from the current viewpoint based on Gaussian primitive projection, spherical harmonic color representation, and transparency accumulation mechanism; Step 24: Convert the Gaussian scene or its point cloud proxy into an Euclidean symbolic distance field (ESDF); Step 25: Construct an ESDF voxel mesh at a preset resolution, and obtain the distance between the UAV's current position and the nearest obstacle through trilinear interpolation during training; Step 26: Sample target points from the ESDF occupied surface, bridge mesh surface, or Gaussian point cloud proxy. Each target sampling point contains spatial coordinates, surface normal vector, minimum observation cosine threshold, and optional semantic category. Step 27: Establish an interface between Gaussian new perspective rendering, ESDF safe distance query, coverage sampling point statistics, and reinforcement learning step() process, so that visual observation, geometric safety information, and coverage reward can be updated synchronously at each time step.

[0012] Furthermore, step three specifically includes: Step 31: Model the UAV autonomous inspection task as a Markov decision process. At each time step, the agent selects an action based on the current observation, the environment updates the UAV state based on physical constraints, and returns the new observation and reward. Step 32: Construct a multimodal observation space composed of visual realism, task progress, and local security; Step 33: Define the action space as a four-dimensional continuous velocity command and use the velocity command as the action output so that the strategy can adapt to velocity limits, inertial response, control delay and underlying flight control constraints during the training phase.

[0013] Furthermore, step four specifically includes: Step 41: Discretize the target structure surface into a set of covering sampling points; Step 42: For any covered sampling point, determine whether the point has been effectively observed based on the UAV camera center position, camera optical axis direction, target point distance, camera field of view angle, and target point surface normal vector; Step 43: Use a persistent overlay mask to record the overlay status. Initially, all overlay points are in an unobserved state. Update the overlay mask at each time step based on the visibility results. Step 44: Construct a frontier exploration score to measure the extent to which the current camera is pointing towards uncovered areas; Steps four and five: Constructing pose alignment rewards; Step 46: Construct semantic priority rewards; Step 47: Construct safety penalties and collision penalties; Step 48: Combine the coverage reward, information gain reward, pose alignment reward, semantic priority reward, stagnation penalty, boundary penalty, safety penalty, and collision penalty into a total reward function.

[0014] Furthermore, step five specifically includes: Step 51: Construct a multimodal Actor-Critic policy network; Step 52: Use SHAC to perform differentiable pre-training on the Actor-Critic policy network; Step 53: Use the PPO method to perform high-fidelity fine optimization on the SHAC pre-trained policy network; Step 54: Employ a domain randomization mechanism to improve policy robustness.

[0015] Furthermore, step six specifically includes: Step 61: Convert the continuous actions output by the policy network into flight control speed commands; Step 62: Establish a closed loop for pose feedback, Gaussian rendering, and time synchronization; Step 63: Conduct strategy verification and engineering usability evaluation in a real-scale bridge simulation environment.

[0016] Furthermore, the multimodal Actor-Critic policy network includes a visual encoding branch, a state encoding branch, a shared feature fusion module, an Actor output head, and a Critic output head. The visual encoding branch processes RGB images, depth maps, or semantic maps obtained from the rendering of a 3D Gaussian scene of the bridge, extracting bridge structural texture, component boundaries, local occlusion, damage appearance, and spatial geometric features. The state encoding branch processes low-dimensional task states, including coverage, recent coverage gain, frontier exploration score, safe distance, attitude alignment score, semantic target completion, remaining range, or flight control feedback state. These two types of features are concatenated and input into a shared multilayer perceptron in the shared feature fusion module for fusion, resulting in a joint feature representation for policy decision-making and value estimation. The Actor output head outputs continuous action distribution parameters based on the joint feature representation, used to generate UAV 3D linear velocity and yaw rate commands. The Critic output head outputs a state value estimate based on the joint feature representation, used to calculate the advantage function, value function loss, and policy update objective.

[0017] The present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the bridge unmanned aerial vehicle autonomous inspection method based on Gaussian scene perception and reinforcement learning.

[0018] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the bridge unmanned aerial vehicle autonomous inspection method based on Gaussian scene perception and reinforcement learning.

[0019] The beneficial effects of this invention are: (1) This invention extends the three-dimensional Gaussian scene from offline mapping results to a reinforcement learning interactive environment for UAV autonomous inspection. It can generate photorealistic RGB images, depth maps and semantic maps in real time based on the current pose of the UAV, reducing the visual domain difference between traditional simulation images and real bridge inspection images.

[0020] (2) This invention decouples Gaussian rendering from dynamics simulation, uses an independent Gaussian rendering microservice to provide visual observation, and performs dynamics state updates by PX4 / Gazebo or a real flight control system, so that the strategy benefits from both the appearance of the real scene and the constraints of real flight dynamics.

[0021] (3) This invention transforms the collision detection and safe distance lookup of complex structural geometry into a voxel lookup table and interpolation problem through the ESDF distance field, which can quickly obtain the closest distance between the UAV and the bridge structure during the high-frequency interaction process of reinforcement learning, thereby improving training safety and efficiency.

[0022] (4) This invention proposes a multi-objective reward function for coverage inspection, which integrates coverage gain, frontier exploration, attitude alignment, semantic priority, safety distance and collision risk into a unified model, so that the strategy not only pursues area coverage, but also prioritizes the observation of piers, supports, key connection areas at the bottom of beams, cable anchorage areas and areas with high incidence of defects.

[0023] (5) This invention uses a smooth visibility kernel to transform the hard visibility judgment in the coverage task into a continuously differentiable form, so that the coverage reward, security constraints and state transition can be incorporated into the computation graph, supporting the application of differentiable strategy optimization methods such as SHAC in the inspection coverage task and improving the training efficiency of small samples.

[0024] (6) This invention uses speed commands as strategy actions and connects to the PX4 flight control system through MAVROS, so that real flight constraints such as speed limits, inertial response, control delay and motor saturation are incorporated into the training closed loop, thereby improving the executability of the strategy from the simulation environment to the real UAV platform.

[0025] (7) In a specific embodiment, the real-scale bridge simulation experiment can achieve 100% detection of 15 damaged targets, 98.5% coverage of the bottom surface of the beam, 100% coverage of the pier surface and no collision throughout the process, proving that the present invention has engineering applicability in terms of realistic visual training, strategy optimization and flight control closed-loop verification. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0027] Figure 1 This is the overall flowchart of the reinforcement learning training method for UAV autonomous inspection based on Gaussian scenarios.

[0028] Figure 2 This is the overall architecture diagram of the bridge 3D Gaussian scene construction process.

[0029] Figure 3 It is a visualization diagram of the overall bridge simulation.

[0030] Figure 4 This is a visualization diagram of bridge damage simulation.

[0031] Figure 5 This is a schematic diagram of the Gaussian rendering microservice and the PX4 / Gazebo flight control closed loop.

[0032] Figure 6 This is a diagram illustrating the architecture of a reinforcement learning task model for autonomous drone inspection.

[0033] Figure 7 This is a schematic diagram of the multi-objective reward function for inspection coverage.

[0034] Figure 8 This is a schematic diagram of a training framework for an autonomous inspection strategy for unmanned aerial vehicles (UAVs) based on short-time domain differentiable pre-training and near-end policy fine-tuning.

[0035] Figure 9 This is a schematic diagram of the multimodal observation input and Actor-Critic decision output process.

[0036] Figure 10 This is a diagram illustrating the closed-loop interaction of reinforcement learning. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Specifically, in combination Figures 1-10 This invention proposes an autonomous bridge inspection method using unmanned aerial vehicles (UAVs) based on Gaussian scene perception and reinforcement learning. The specific steps include: Step 1: Construct a 3D Gaussian training scene for UAV autonomous inspection to represent the texture, geometry, depth, and semantic information of the bridge structure.

[0039] Step 2: Establish a Gaussian rendering and geometric safety constraint module to generate UAV visual observations in real time and complete safe distance queries, collision judgments, and coverage statistics.

[0040] Step 3: Construct a reinforcement learning task model for UAV autonomous inspection, taking visual observation, task status, local geometric information and flight control status as inputs, and speed commands as action outputs.

[0041] Step 4: Design a multi-objective reward function for coverage inspection to uniformly evaluate coverage gain, information gain, pose alignment, semantic priority, security risks, and invalid behaviors.

[0042] Step 5: Train the UAV autonomous inspection policy network using the near-end policy optimization method and the short-time domain differentiable policy optimization method, and improve the policy robustness through the domain randomization mechanism.

[0043] Step Six: Integrate the trained UAV autonomous inspection strategy into the flight control closed-loop system to achieve closed-loop verification, flight execution, and engineering deployment of the inspection strategy.

[0044] Step one involves constructing a 3D Gaussian training scene for autonomous drone inspection, used to represent the texture, geometry, depth, and semantic information of the bridge structure. The overall architecture of this step is as follows: Figure 2 As shown, the specific steps include: Step 11: Acquire multi-source scene data of the inspection object. Due to the characteristics of large-scale bridge scenes at the real engineering scale, such as large structural scale, high acquisition costs, complex flight approval processes, and difficulties in data annotation, there is a lack of publicly available multi-view datasets of large bridges that can be directly used for reinforcement learning training of UAV autonomous inspection. To verify the effectiveness of the proposed method at a real engineering scale, an existing real-scale bridge (the elevated approach section of the Tsing Sha Highway Viaduct in Hong Kong) in GLB format digital 3D model was used as the basic scene data, and it was converted into a bridge model that can be loaded into the Gazebo simulation environment.

[0045] Specifically, taking the digital model of the continuous steel box girder structure of the elevated approach section of the Tsing Sha Highway Bridge in Hong Kong as an example, a bridge simulation scene was constructed based on its real-scale model, and damage targets such as cracks, spalling or corrosion were arranged on the bottom of the beams and the pier area.

[0046] When the coordinate system of the original GLB model is inconsistent with the coordinate system of the simulation environment, coordinate system transformation, scale preservation, and centering are performed on the bridge model to enable it to be loaded at a real engineering scale in the simulation environment. The bridge model maintains its real physical dimensions without scaling to ensure that the bridge span, pier height, beam bottom elevation, component spacing, and UAV flight safety distance all have engineering dimensional significance.

[0047] In a bridge simulation environment, multi-view RGB images and corresponding precise camera poses are acquired using a virtual camera from the bottom of the beam, the sides of the bridge, the top of the bridge deck, the sides of the piers, and near key components. Furthermore, damage patches are placed on the bottom of the bridge beam, the surface of the piers, near the supports, in the cable anchorage area, or other key component areas to simulate cracks, spalling, corrosion, or other localized defects. These damage patches can be attached to the bridge surface as transparent textures or thin patches, ensuring that damage annotations are decoupled from the collision geometry of the main bridge structure.

[0048] Steps 1 and 2: Establish bridge scene pose constraints and initialize 3D Gaussian primitives. Use the precise camera pose output from the virtual camera in the Gazebo simulation environment as the pose input for training the 3D Gaussian scene, avoiding scene reconstruction drift caused by pose estimation errors. Subsequently, initialize Gaussian primitive positions using point clouds sampled from the bridge mesh model, multi-view reconstructed point clouds, or simulated depth backprojection point clouds. Each Gaussian primitive includes at least a 3D center position, covariance matrix, opacity, color, or spherical harmonic coefficients, and may further include semantic features. The 3D Gaussian scene is represented as follows:

[0049] In the formula, Indicates the first The three-dimensional center position of each Gaussian element. Represents the covariance matrix. Indicates opacity. Indicates color or spherical harmonic coefficient. Represents semantic features, The number of Gaussian elements represents the number of Gaussian elements. Compared to traditional mesh or voxel models, 3D Gaussian scenes can express the realistic texture, geometric details, and semantic properties of structural surfaces in a continuous, compact, and differentiable form.

[0050] Step 13: Jointly optimize the 3D Gaussian scene parameters of the bridge. Utilize multi-view RGB image photometric reconstruction loss, structural similarity loss, simulation depth supervision term, geometric anchoring constraints, and depth correlation constraints to jointly optimize the Gaussian meta-parameters, ensuring that the rendered image maintains consistency with the bridge simulation observations in terms of color, structure, depth, and geometric distribution. The optimization objective is:

[0051] In the formula, , , and These represent the weight coefficients of the semantic constraint, depth constraint, geometric anchoring constraint, and pose regularization constraint, respectively. Through the above optimization, a realistic-scale 3D Gaussian training scene of a bridge is generated, including the beam bottom, piers, bridge sides, supports, cable anchorage zones, and damage patch areas. This scene can render RGB images, depth maps, and semantic maps in real time based on the current camera pose of the UAV, providing a visual observation foundation with realistic bridge structural features for subsequent reinforcement learning training.

[0052] Step 14: Construct additional scene information for bridge inspection tasks. Based on the bridge Gaussian scene, bridge mesh model, or bridge sampling point cloud, construct an Euclidean symbolic distance field (ESDF). Transform the distance query between the UAV and the bridge structural surface into a voxel mesh lookup table problem for subsequent collision detection and safety distance penalty calculation. Subsequently, sample target points from the beam bottom, piers, supports, cable anchorage zones, or other key component surfaces. Record the 3D coordinates, surface normal vector, minimum observation angle threshold, and semantic category for each covered target point for subsequent coverage calculation, attitude alignment reward, and semantic priority reward.

[0053] Furthermore, multiple types of virtual camera viewpoints are automatically planned in the bridge scene, including a dense scanning viewpoint at the bottom of the beam, a bird's-eye viewpoint above the bridge deck, a transverse crossing scanning viewpoint, and a close-up surround shooting viewpoint of damaged targets, to enhance the reconstruction quality of the bridge Gaussian scene in key components and defect areas. An example of the overall bridge simulation scene is shown below. Figure 3 As shown, an example of a bridge damage simulation scenario is as follows: Figure 4 As shown.

[0054] Step two involves establishing a Gaussian rendering and geometric safety constraint module to generate real-time UAV visual observations and complete safe distance queries, collision detection, and coverage statistics. This includes the following steps: Step 21: Before reinforcement learning training begins, the 3D Gaussian scene model is loaded into a separate rendering process, making the Gaussian model resident in GPU memory and providing on-demand rendering services via HTTP, RPC, or other lightweight interfaces. This design avoids repeatedly loading Gaussian scene parameters in the reinforcement learning step() loop, reducing GPU memory contention and rendering blocking during multi-process training.

[0055] Step 22: At each time step, the physics simulator or flight control system outputs the UAV's six-degree-of-freedom pose in the global coordinate system:

[0056] In the formula, Indicates the location of the drone. This represents the quaternion of the UAV's attitude. The system uses camera extrinsic parameters... The drone system pose is converted into a camera pose, and the camera pose is then passed to the Gaussian rendering backend.

[0057] Steps 2 and 3: The Gaussian rendering backend generates an RGB image, depth map, and semantic map from the current viewpoint based on Gaussian primitive projection, spherical harmonic color representation, and transparency accumulation mechanism. For image pixel u, the color synthesis process is represented as follows:

[0058] In the formula, Represents pixels The rendered colors, Indicates the affected pixels Gaussian set, Indicates the first i Each Gaussian pixel The contribution of opacity This represents the color or spherical harmonic color value of the Gaussian element. This represents the transmittance of the foreground Gaussian to the background Gaussian. This point-based rasterization process can be executed in parallel on a GPU, making it suitable for generating high-frequency visual observations during reinforcement learning training.

[0059] Step 24: Convert the Gaussian scene or its point cloud proxy into an Euclidean symbolic distance field (ESDF). For cases where the original scene is large, spatially prune the point cloud based on the acquisition trajectory bounding box or a preset flight area, retaining only the structural surface points related to the UAV inspection path to reduce the number of voxels and storage overhead.

[0060] Step 25: Construct an ESDF voxel mesh at a preset resolution, and obtain the distance between the drone's current position and the nearest obstacle during training using trilinear interpolation.

[0061] In the formula, This represents the distance from the drone to the nearest obstacle at time step t. This distance is used for safety penalties, collision termination judgments, and local obstacle avoidance observation.

[0062] Step 26: Sample target points from the ESDF occupied surface, bridge mesh surface, or Gaussian point cloud proxy. Each target sampling point includes spatial coordinates, surface normal vector, minimum observation cosine threshold, and optional semantic category. Perform spatial range filtering and normal filtering on the sampling points, retaining only target points located near the flyable area that can be observed directly or nearly directly by the UAV camera.

[0063] Step 27: Establish interfaces between Gaussian new perspective rendering, ESDF safe distance query, coverage sampling point statistics, and the reinforcement learning step() process, enabling visual observation, geometric safety information, and coverage reward to be updated synchronously at each time step. The relationship between the Gaussian rendering microservice and the flight control closed loop is as follows: Figure 5 As shown.

[0064] Step three involves constructing a reinforcement learning task model for autonomous UAV inspection, using visual observation, task status, local geometric information, and flight control status as inputs, and speed commands as action outputs. For example... Figure 6 As shown, the specific steps include: Step 31: Model the UAV autonomous inspection task as a Markov decision process:

[0065] In the formula, Representing the state space, Represents the action space, Represents the state transition function. Represents the reward function, This represents the discount factor. At each time step, the agent selects an action based on the current observation, the environment updates the drone's state according to physical constraints, and returns the new observation and reward.

[0066] Step 32: Construct a multimodal observation space composed of visual realism, task progress, and local safety. The observation at time step t is defined as follows:

[0067] In the formula, This represents the visual observation obtained from rendering a 3D Gaussian scene. This represents the global task state vector. This represents the local geometric distance feature.

[0068] The visual observation It includes RGB images from the current camera's perspective, which can be further stitched together with depth maps and semantic maps. RGB images provide information on bridge surface texture, component boundaries, and lighting; depth maps represent the geometry and relative distances from the current perspective; and semantic maps identify piers, supports, beam bottoms, cracks, damage patches, or other key areas. All three types of visual information originate from the same Gaussian scene, thus maintaining consistency in spatial projection relationships.

[0069] The global task state vector This includes current global coverage, single-step coverage gain, frontier exploration score, semantic target completion, camera optical axis and target surface attitude alignment score, minimum safe distance obtained from ESDF query, remaining range or remaining time budget, and safety violation flag, represented as:

[0070] In the formula, Indicates cumulative coverage. Indicates single-step coverage gain. Indicates the score for cutting-edge exploration. Indicates the degree of semantic goal completion. This indicates the posture alignment score. Indicates the minimum safe distance. Indicates the remaining flight distance or time budget. Indicates a safety violation sign.

[0071] The local geometric distance features This includes distance sector vectors from a one-dimensional LiDAR array, local point cloud projection, or multi-directional nearest obstacle distances obtained from ESDF queries, expressed as:

[0072] In the formula, This represents the distance to the nearest obstacle within the i-th sector. This indicates the total number of directional sectors. This feature provides a higher frequency and more direct obstacle avoidance signal than visual images, compensating for the uncertainties in visual observation under conditions of occlusion, weak texture, or motion blur.

[0073] Step 33: Define the motion space as a four-dimensional continuous velocity command:

[0074] In the formula, These represent the target linear velocities of the UAV in three-dimensional space, respectively. This represents the target yaw rate. The speed command is used as the action output, allowing the strategy to adapt to speed limits, inertial response, control delay, and underlying flight control constraints during the training phase.

[0075] Step four involves designing a multi-objective reward function for coverage inspection, which uniformly evaluates coverage gain, information gain, pose alignment, semantic priority, security risks, and invalid behaviors. This includes the following steps: Step 41: Discretize the target structure surface into a set of covering sampling points:

[0076] In the formula, Indicates the firsti Coordinates of surface sampling points This represents its surface normal vector. Represents the minimum observed cosine threshold. Indicates semantic category, M This indicates the total number of sampling points covered.

[0077] Step 42: For any sampling point, determine whether the point has been effectively observed based on the UAV camera center position, camera optical axis direction, target point distance, camera field of view angle, and target point surface normal vector. The conditions for effective observation are expressed as:

[0078] In the formula, Indicates the distance from the camera center to the 1st. i The distance between each sampling point Indicates the maximum effective observation distance. Indicates the direction of the camera's optical axis. This represents the unit direction vector pointing from the camera center to the sampling point. θ Indicates the camera's field of view. This represents the surface normal vector. When all three conditions within the parentheses are met simultaneously... ,otherwise 0. Distance constraints ensure that the target point is within the effective detection range; field of view constraints ensure that the target point falls within the camera's imaging range; normal constraints ensure that the UAV observes from the front or near-front direction of the structure's surface.

[0079] Step 43: Use a persistent overlay mask to record the overlay status. Initially, all overlay points are in an unobserved state. Update the overlay mask at each time step based on the visibility results.

[0080] In the formula, Indicates the first i The coverage status of each sampling point at time step t This represents the cumulative coverage at time step t. For visibility indicators, This represents the total number of covered sampling points. The coverage bonus is calculated based on the positive increment of coverage rate between adjacent time steps:

[0081] The reward is only given to newly covered areas to prevent the agent from repeatedly wandering in the already observed areas.

[0082] Step 44: Construct a frontier exploration score to measure the degree to which the current camera is pointing towards uncovered areas. During the high coverage phase, even if no new coverage points are generated at the current time step, a directional reward is still given as long as the camera is still pointing towards potentially uncovered areas, to alleviate the reward sparsity problem in the later stages of the coverage task. The information gain reward is represented as:

[0083]

[0084]

[0085] In the formula, The information gain reward represents the reward at time step t. This represents the average positive coverage gain over the most recent L time steps. This indicates the current camera's exploration score towards the uncovered area. and For adjustment coefficients, C t This represents the cumulative coverage at time step t. L Indicates the length of the statistical window. M Indicates the total number of covered sampling points. Indicates the first i The cumulative coverage status of each coverage sampling point at time step t Indicates the direction of the camera's optical axis at time step t. Indicates pointing from the center of the camera to the first i Unit direction vector covering each sampling point, This indicates the number of currently uncovered sampling points.

[0086] Steps 4 and 5: Constructing attitude alignment rewards. For the set of surface points within the current field of view that satisfy the visibility constraints, calculate the angle between the camera optical axis and the target direction, as well as the angle between the target surface normal vector and the observation direction. This allows the UAV to align with the bridge component surface at a more reasonable angle, reducing the impact of perspective distortion, grazing observation, and local occlusion on the quality of the detection image.

[0087] Step 46: Construct a semantic priority reward. Utilize existing semantic features in the 3D Gaussian scene, and pass the semantic category of Gaussian elements to the coverage sampling points through nearest neighbor association, voxel inheritance, or projection raying. Assign a higher inspection priority than the general background area to piers, supports, critical connection areas at the bottom of beams, cable anchorage areas, damage patches, and areas with high incidence of defects. The semantic priority reward is represented as follows:

[0088] In the formula, Indicates the first cInspection priority weights for semantic regions This indicates the increased effective coverage of the semantic category at the current time step. This reward shifts the strategy from simply pursuing area coverage to prioritizing coverage of critical bridge components and areas prone to defects.

[0089] Step 47: Construct safety penalties and collision penalties. Continuous safety penalties are based on the nearest obstacle distance obtained from an ESDF query. calculate:

[0090] In the formula, This represents the safe distance threshold. When the drone is within the safe range, no penalty is incurred; as the drone approaches the structure's surface, the penalty increases with decreasing distance. If... Less than the collision threshold If this happens, a collision penalty will be triggered and the current training round will be terminated.

[0091] Step 48: Combine the coverage reward, information gain reward, pose alignment reward, semantic priority reward, stagnation penalty, boundary penalty, safety penalty, and collision penalty into a total reward function:

[0092] In the formula, , , , , , , and These are the weight coefficients for each reward or penalty item. The multi-objective reward function is composed as follows: Figure 7 As shown.

[0093] Step five involves training the UAV autonomous inspection policy network using a near-end policy optimization method and a short-time domain differentiable policy optimization method, and improving policy robustness through a domain randomization mechanism. For example... Figure 8 As shown, the specific steps include: Step 51: Construct a multimodal Actor-Critic policy network, specifically including: This step is used to establish a strategy network that can simultaneously process Gaussian visual observations, mission status information, and flight feedback information, enabling the UAV to output continuous flight actions based on the current bridge inspection status.

[0094] Specifically, such as Figure 9As shown, the multimodal Actor-Critic policy network includes a visual encoding branch, a state encoding branch, a shared feature fusion module, an Actor output head, and a Critic output head. The visual encoding branch processes RGB images, depth maps, or semantic maps obtained from the bridge's 3D Gaussian scene rendering, extracting bridge structural texture, component boundaries, local occlusion, damage appearance, and spatial geometric features. The state encoding branch processes low-dimensional task states, including coverage, recent coverage gain, front-end exploration score, safe distance, attitude alignment score, semantic target completion, remaining range, or flight control feedback state. The two types of features are concatenated and input into a shared multilayer perceptron for fusion, obtaining a joint feature representation for policy decision-making and value estimation. The Actor output head outputs continuous action distribution parameters based on the joint feature representation, used to generate UAV 3D linear velocity and yaw rate commands; the Critic output head outputs a state value estimate based on the joint feature representation, used to calculate the advantage function, value function loss, and policy update objective. Thus, the policy network can simultaneously complete inspection action generation and current state value assessment within the same framework.

[0095] Step 52: Use SHAC to perform differentiable pre-training on the Actor-Critic policy network.

[0096] This step is used to improve the efficiency of policy pre-training under differentiable simulation conditions, so that the policy network can obtain basic coverage inspection capabilities before entering the high-fidelity 3D Gaussian rendering environment and flight control closed-loop environment.

[0097] To address the issue of non-differentiability of hard visibility judgments in coverage tasks, a smooth visibility kernel based on the sigmoid function is introduced to transform discrete binary visibility or invisibility judgments into continuously differentiable forms:

[0098] In the formula, σ This represents the sigmoid function. , and This is a temperature parameter used to control the degree of closeness between the soft approximation and the hard threshold; Indicates the drone camera to the number i The distance between each covered sampling point Indicates the maximum effective observation distance; Indicates the direction of the camera's optical axis at time t. This represents the unit direction vector pointing from the camera center to the i-th covered sampling point. θ Indicates the camera's field of view; Indicates the first i The surface normal vector covering the sampling points, This represents the minimum normal observation threshold.

[0099] Through this smooth visibility kernel, coverage rewards, safety distance penalties, and dynamic state transitions can be incorporated into a differentiable computational graph. Based on this, the short-time domain Actor-Critic (SHAC) method is used for differentiable pre-training of the policy network. SHAC accumulates reward gradients through direct backpropagation using short-time domain differentiable trajectory expansion and estimates long-term values ​​after the short-time domain using Critic, thereby reducing gradient variance and credit assignment difficulty in long-time domain coverage inspection tasks.

[0100] During training, the policy network outputs continuous velocity actions based on the current Gaussian visual observations, low-dimensional task state, and local geometric distance features. The differentiable environment calculates the short-term cumulative reward based on the UAV motion model, soft coverage reward, and safe distance constraints, and backpropagates the gradient to the Actor network parameters. Simultaneously, the Critic network learns long-term value estimates after truncating the temporal domain to compensate for the inability of short-term expansion to cover the complete episode. After training, the Actor-Critic policy network parameters with basic coverage inspection capabilities are obtained and used as initialization parameters for subsequent PPO high-fidelity fine-tuning.

[0101] The following section, based on specific experimental results, explains the effectiveness of using SHAC for differentiable pre-training of the Actor-Critic policy network in step 5.2 in improving training efficiency and inspection policy performance.

[0102] In this embodiment, a realistic 3D Gaussian scene of the Tsing Sha Highway Viaduct in Hong Kong is selected as the source bridge verification scene. During training, the 3D Gaussian scene of the bridge provides RGB visual observations to the reinforcement learning environment through a rendering service, enabling the policy network to make inspection decisions with the participation of realistic visual input. This experiment verifies the effectiveness of Gaussian rendering visual observation, differentiable dynamics modeling, and the SHAC policy optimization method in real-world bridge visual inspection tasks.

[0103] During policy training, the same Gaussian bridge scene, RGB visual rendering input, policy network skeleton, and coverage rewards and safety constraints were used. Only the policy gradient optimization method was changed, and the training efficiency of SHAC, BPTT (Backpropagation Through Time), and PPO was compared. SHAC accumulates reward gradients through direct backpropagation of short-time-domain differentiable trajectories and uses Critic estimation to determine the long-term value after truncation in the time domain. BPTT optimizes the unfolded trajectory through time-series backpropagation, exhibiting strong early local optimization capabilities. PPO serves as a model-free reinforcement learning baseline, updating the policy through trajectory sampling, advantage estimation, and pruning the objective function. All three methods were trained with visual rendering input to compare the convergence efficiency, sample utilization efficiency, and final performance of different optimization methods in real-world bridge visual inspection tasks.

[0104] Specifically, in the experiment, the environment batch size was set to 8, the training time unrolling steps for SHAC were set to 16, the training time unrolling steps for BPTT were set to 32, and the sampling steps before policy update for PPO were set to 16. During training, the system adopted a dual-GPU approach, with one GPU used for policy optimization and the other for Gaussian scene rendering to ensure stable operation of visual observation generation and policy update. The training efficiency comparison of different methods is shown in Table 1.

[0105] Table 1. Performance Comparison of Different Strategy Optimization Methods in Real-World Bridge Visual Inspection Tasks

[0106] As shown in Table 1, in the realistic bridge inspection task with RGB visual rendering input, SHAC outperforms BPTT and PPO in both final average reward and best average reward metrics. This indicates that SHAC can more fully utilize the Gaussian scene and differentiable dynamic structure of the real bridge, achieving higher quality policy updates with a limited sample budget. BPTT achieves better results in the early iteration stages, indicating its strong local trajectory optimization capability, but its final performance is lower than SHAC, suggesting relatively insufficient stability in later stages. Although PPO can complete the training process, its final average reward and best average reward are significantly lower than the two differentiable optimization methods with similar sample sizes, indicating that the sample utilization efficiency of the sampling-based model-free policy gradient method is relatively low in this type of visual input, continuous control, and long temporal coverage task.

[0107] The above results demonstrate that, in real-world bridge Gaussian visual inspection tasks, combining RGB Gaussian rendering visual observation, smoothed visibility kernels, short-term differentiable trajectory unfolding, and Critic value estimation can reduce gradient variance and credit assignment difficulty in long-term coverage inspection tasks, thereby improving the training efficiency of UAV visual inspection strategies. Therefore, in the subsequent complete strategy training process, SHAC is used as a differentiable pre-training method to first obtain the parameters of the Actor-Critic strategy network with basic coverage inspection capabilities; subsequently, PPO is used for stable fine-tuning in a high-fidelity bridge 3D Gaussian rendering environment to obtain the base strategy network used for transfer experiments.

[0108] Step 53: Use the PPO method to perform high-fidelity fine optimization on the SHAC pre-trained policy network.

[0109] After obtaining the initial policy parameters of the Actor-Critic pre-trained SHAC, the policy network is connected to the bridge 3D Gaussian rendering visual observation, ESDF safety distance constraints and continuous velocity motion space, and the near-end policy optimization algorithm PPO is used to further train the policy network.

[0110] Let r be the probability ratio between the old and new strategies. t (θ), the objective function for PPO pruning is expressed as:

[0111] in:

[0112] In the formula, A t This represents the generalized advantage estimation. ε This represents the clipping threshold, and clip(·) represents the clipping function. The probability ratio between the old and new strategies. PPO pruning threshold, As the current strategy, For the old strategy, For the action at time step t, This is a multimodal observation for time step t. The pruning mechanism limits the magnitude of each policy update, preventing drastic changes in the policy distribution and thus improving training stability.

[0113] Furthermore, a value function loss and entropy regularization term are added to the PPO objective function to maintain a certain level of exploratory capability during training and improve the value estimation quality of Critic for long-term coverage inspection tasks. For bridge coverage inspection tasks, since the duration of a single episode is relatively long, early actions can have a delayed impact on later coverage rates. Therefore, the number of sampling steps is set to a scale close to the episode length, ensuring that each policy update covers the complete inspection trajectory, thereby alleviating the difficulty of credit allocation in long-term coverage tasks.

[0114] In this step, PPO no longer starts training from a randomly initialized policy. Instead, it loads the Actor-Critic policy parameters obtained through SHAC pre-training in step 5.2 as initialization parameters. Thus, SHAC is used to quickly obtain basic coverage inspection capabilities, while PPO is used to stably fine-tune the policy under high-fidelity Gaussian visual observation, ESDF safety constraints, and continuous velocity control conditions, making the training process take into account sample efficiency, visual realism, and engineering deployability.

[0115] Step 54: Employ a domain randomization mechanism to improve policy robustness.

[0116] In the SHAC differentiable pre-training and PPO high-fidelity fine-tuning process, a domain randomization mechanism is introduced. At the beginning of each round, parameters such as aircraft mass, maximum thrust, moment of inertia, motor response delay, air drag coefficient, control delay, initial position perturbation, initial attitude perturbation, lighting conditions, camera noise, image blur, rendering brightness perturbation, and depth noise are randomly sampled to ensure that the strategy maintains stable performance under different dynamic conditions, visual conditions, and sensor perturbation conditions.

[0117] Through the aforementioned domain randomization mechanism, the policy network can be exposed to various dynamic disturbances, visual disturbances, and sensor errors during the training phase, thereby reducing its dependence on a single bridge appearance, a single lighting condition, and an idealized UAV dynamics model, and improving the robustness of the UAV autonomous inspection strategy and its ability to transfer from simulation to reality.

[0118] The following specific embodiments illustrate the implementation effects of training the UAV autonomous inspection strategy network using a combination of SHAC differentiable pre-training and PPO high-fidelity fine-tuning in step five, and the results achieved through domain randomization in improving training efficiency, inspection coverage performance, and cross-scenario robustness.

[0119] This embodiment first evaluates the inspection performance of the Actor-Critic base strategy obtained after SHAC differentiable pre-training and PPO high-fidelity fine-tuning in the source bridge 3D Gaussian scene; then, the base strategy is transferred to the target bridge 3D Gaussian reconstruction scene, and strategy transfer and fine-tuning experiments are carried out to verify the adaptability and fine-tuning effect of the method in the new bridge scene.

[0120] First, in the 3D Gaussian scene of the Tsing Sha Highway Viaduct in Hong Kong, the pedestal policy, pre-trained with SHAC differentiability and fine-tuned by PPO with high fidelity, was evaluated in independent rounds. During the evaluation, a deterministic policy was used to output UAV actions. The policy network output four-dimensional continuous velocity commands based on the Gaussian-rendered RGB image and the low-dimensional task state vector. The environment called the Gaussian rendering service to generate the next moment's visual observation based on the UAV's motion state. Coverage reward, safety penalty, collision state, minimum safe distance, and final coverage were calculated using a coverage tracker and ESDF distance field. The coverage inspection performance of the pedestal policy in the source bridge 3D Gaussian scene was obtained, serving as the source scene performance benchmark for subsequent target scene migration experiments. The performance results of the pedestal policy in the source bridge 3D Gaussian scene are shown in Table 2.

[0121] Table 2 Performance of the base strategy in the 3D Gaussian scene of the source bridge

[0122] The strategy achieved final coverage rates of 78.57%, 78.57%, and 100.00% in three independent rounds of the source bridge Gaussian scenario, with an average coverage rate of 85.71%. Simultaneously, no collisions occurred during the three evaluations, and the minimum safe distance remained at 0.100m. These results indicate that the strategy has been able to establish relatively stable coverage inspection behavior in the Qingsha Highway Bridge Gaussian scenario, maintaining basic safety while achieving high coverage. Specifically, the 100.00% coverage rate achieved in round 3 demonstrates that under suitable initial conditions and trajectory evolution, the strategy can complete a full inspection of the target area; the stable coverage rate of 78.57% in the other two rounds indicates that the strategy still possesses high surface coverage capability under different initial conditions.

[0123] Furthermore, to verify the transfer capability and fine-tuning effect of the described strategy training method in the new bridge 3D Gaussian scene, a strategy transfer and fine-tuning experiment for the target bridge Gaussian scene was set up in the strategy training process corresponding to step five. The experiment used the PPO Actor-Critic policy network, trained in the source bridge Gaussian scene, as the base model and transferred it to the target bridge 3D Gaussian reconstruction scene for further training. The target bridge scene data comes from the Photogrammetry Data for VTTI Bridge (CIAMTIS) dataset released by Virginia Tech. This dataset uses a multi-span composite structure bridge in the Virginia Smart Roads test field as the object, including typical inspection areas such as the concrete bridge deck, piers, cap beams, steel main beams, bridge sides, and under-bridge space, providing a real engineering scenario for bridge 3D reconstruction and UAV inspection strategy verification. The target bridge 3D Gaussian reconstruction scene was reconstructed from the bridge real-scene images in this dataset using the 3D Gaussian splashing method, and has structural features such as non-convex arches, multiple piers, and complex bridge boundaries, differing from the source bridge scene in geometric layout and visual appearance.

[0124] During the fine-tuning of the target bridge scenario, adjustments were made to the coverage sampling density, ESDF safety constraint accuracy, and PPO temporal backtracking length. Specifically, the coverage grid sampling step size was adjusted from 6 to 3 to increase the number of bridge surface coverage sampling points and reduce the fluctuations in coverage statistics caused by the omission of a single sampling point; the ESDF point cloud step size was adjusted from 8 to 4, and the geometric voxel resolution was adjusted from 0.24m to 0.16m to improve the accuracy of safety distance queries near the bridge structure boundary; at the same time, the maximum number of physical steps per round was increased from 180 to 360, and the PPO trajectory sampling step size was synchronously set to 360, enabling policy updates to cover the entire inspection temporal domain and alleviating the difficulty of credit allocation in long temporal coverage tasks.

[0125] During fine-tuning, the policy network outputs a four-dimensional continuous velocity command based on the Gaussian-rendered RGB image and the low-dimensional task state vector. The environment, based on the UAV's motion state, calls the Gaussian rendering service to generate the visual observation for the next moment, and calculates the coverage reward and safety penalty through the coverage tracker and ESDF distance field. After fine-tuning on the target bridge scene, a deterministic policy is used to conduct independent episode evaluations at a standard starting point, and the results are shown in Table 3.

[0126] Table 3. Strategy transfer results for different scenarios

[0127] Table 3 shows that when the source scene base strategy was directly transferred to the target bridge's 3D Gaussian reconstruction scene, the final coverage rates for the three rounds were 57.14%, 67.86%, and 42.86%, respectively, with an average coverage rate of 55.95% ± 12.54%. This result indicates that while the inspection strategy learned from the source bridge scene possesses a certain basic coverage capability, it still exhibits significant adaptation limitations when facing the new geometry and visual appearance of the target bridge. In particular, the coverage rate fluctuated considerably between the three rounds, suggesting that the direct transfer strategy is highly sensitive to the initial state, local structural boundaries, and visible area distribution in the target scene, and cannot consistently achieve high coverage inspection rates.

[0128] After fine-tuning the PPO in the target bridge scenario, the final coverage rates for the three independent rounds improved to 92.9%, 89.3%, and 85.7%, respectively, with an average coverage rate of 89.30% ± 2.90%. Compared to before the fine-tuning, the average coverage rate increased by an absolute 33.35 percentage points, while the standard deviation of the coverage rate decreased from 12.54% to 2.90%, indicating that the fine-tuning in the target scenario not only significantly improved the final coverage level but also significantly enhanced the stability of the strategy in the target bridge scenario. The fine-tuned strategy achieved a coverage rate of over 85% in all three rounds, demonstrating that it can reliably adapt to the structural features of the target bridge, such as non-convex arches, multiple piers, and complex bridge boundaries.

[0129] The above results demonstrate that the Actor-Critic policy network, after SHAC differentiable pre-training and PPO high-fidelity fine-tuning, can form a basic coverage inspection capability in the source bridge's 3D Gaussian scene. However, when it is directly transferred to the target bridge's 3D Gaussian reconstruction scene with different structural layout and visual appearance, the policy performance is somewhat affected. By continuing to use PPO for fine-tuning in the target scene, and combining higher-density coverage sampling, more refined ESDF safety constraints, and longer-term trajectory sampling settings, the migration adaptability and coverage integrity of the UAV inspection policy in new bridge scenes can be effectively improved. This experiment further illustrates that the UAV autonomous inspection reinforcement learning training method based on Gaussian scenes can not only learn inspection policies in the source scene but also adapt to new bridge structural environments through fine-tuning in the target scene.

[0130] Step six involves integrating the trained UAV autonomous inspection strategy into the flight control closed-loop system to achieve closed-loop verification, flight execution, and engineering deployment of the inspection strategy. Figure 10 A diagram illustrating the closed-loop learning interaction.

[0131] Step 61: Convert the continuous actions output by the strategy network into flight control speed commands.

[0132] This step is used to connect the high-level inspection actions output by the reinforcement learning strategy to the real flight control protocol stack, so that the UAV's movement is no longer directly modified by the reinforcement learning environment, but is executed by the flight control system and physics engine according to real dynamic constraints.

[0133] Specifically, the policy network outputs continuous actions at each time step, the reinforcement learning environment encapsulates these actions as velocity setpoints, and sends them to the PX4 flight control system via the ROS / MAVROS communication interface, for example, by issuing velocity control commands through topics such as / mavros / setpoint_velocity / cmd_vel_unstamped or / mavros / setpoint_raw / local.

[0134] After receiving the speed setpoint, the PX4 flight control firmware calculates the required attitude, angular velocity, and motor speed of the drone through its internal controller. Then, the Gazebo physics engine or the actual drone motor system completes the physics-level calculation. As a result, real flight constraints such as the drone's maximum speed, maximum acceleration, motor saturation, air resistance, control latency, and inertial response can naturally enter the state transition process, giving the trained inspection strategy better physical executability and a better foundation for engineering deployment.

[0135] Step 62: Establish a closed loop for pose feedback, Gaussian rendering, and time synchronization.

[0136] This step ensures that the actual motion state of the UAV, visual observation, safe distance, and reward calculation are consistent within the same time step, thereby forming a closed-loop training and verification system for bridge inspection tasks.

[0137] Specifically, the external flight control system feeds back the UAV's current six-DOF pose to the reinforcement learning environment via pose or odometry. Based on this pose feedback, the reinforcement learning environment invokes the bridge's 3D Gaussian rendering microservice to generate an RGB image, depth map, and semantic map from the camera's perspective at the next moment. Simultaneously, the system queries the ESDF distance field based on the UAV's current position to obtain the minimum safe distance between the UAV and the bridge structure, and combines this with the status of the covered target points to calculate coverage, semantic target completion, and the current reward.

[0138] To avoid inconsistencies in velocity commands, pose feedback, rendered images, and reward calculations in a multi-process system, a timestamp-based cross-process synchronization mechanism is established. After each execution of `step()` by the environment, the system first issues a velocity command and then waits for pose feedback for the corresponding control cycle. Subsequently, the updated pose is sent to the Gaussian rendering service, and the safe distance query and reward calculation are completed simultaneously. Only after the pose, visual image, safe distance, and task state have all been updated is the environment packaged into the next time step observation and input into the policy network. If any modality does not return within a specified time, the environment performs a safe hover, repeats the previous action, or terminates the round to prevent the UAV from continuing to make decisions under incomplete observation conditions.

[0139] Step 63: Conduct strategy verification and engineering usability evaluation in a real-scale bridge simulation environment.

[0140] This step is used to verify the feasibility of the flight control closed loop, inspection coverage, and flight safety of the trained UAV inspection strategy or verification trajectory in a real-world engineering-scale bridge scenario.

[0141] Specifically, a PX4 SITL and MAVROS communication module was integrated into the Gazebo simulation environment for a real-scale bridge, enabling UAVs to perform inspection tasks near the bridge beam bottom, piers, bridge sides, and key components. During verification, a yaw point strategy and a dual-height inspection strategy were combined: the yaw point strategy was used to make the UAV fly along the side of the bridge structure, avoiding direct path crossings of pier collision points; the dual-height inspection strategy was used to allow the UAV to cover the beam bottom and high-level pier areas at a normal altitude, and then descend to a lower altitude for hovering inspection near low-level pier targets, thus addressing the inspection needs of both the beam bottom and low-level pier areas.

[0142] Finally, the training strategy or validation trajectory is evaluated based on the damage target detection rate, beam bottom coverage, pier coverage, flight time, minimum safe distance, and number of collisions. Specifically, the damage target detection rate measures whether preset cracks, spalling, or corrosion targets are effectively observed; beam bottom coverage and pier coverage measure the integrity of inspections of key bridge components; flight time evaluates task execution efficiency; and minimum safe distance and number of collisions evaluate flight safety. These indicators allow for a comprehensive assessment of the engineering usability and deployment potential of the proposed UAV autonomous inspection strategy in a real-scale bridge scenario.

[0143] The following describes, with reference to a specific embodiment, the implementation effect of integrating the trained UAV autonomous inspection strategy into the PX4 / Gazebo flight control closed-loop system in step six, and realizing strategy closed-loop verification, flight execution, and engineering availability assessment through speed command conversion, pose feedback synchronization, Gaussian rendering observation update, and inspection index evaluation.

[0144] In this embodiment, a PX4 / Gazebo flight control closed-loop verification platform for a real-scale bridge is constructed. This platform consists of Gazebo, PX4 SITL, and MAVROS. Gazebo loads the real-scale bridge model and provides a physical simulation environment, PX4 SITL simulates the UAV flight control firmware, and MAVROS enables communication between the ROS environment and the PX4 flight control system. During the verification process, the continuous actions output by the policy network are converted into flight control speed commands, and combined with pose feedback, Gaussian rendering observation, safe distance query, and reward calculation, a complete flight control closed-loop verification process is formed.

[0145] To improve the feasibility of inspections in realistic bridge scenarios, a yaw point strategy and a dual-altitude inspection strategy were combined during the verification process. This enabled the UAV to fly safely along the side of the bridge while simultaneously meeting the observation needs of targets at the bottom of the beams, high-level piers, and low-level piers. Finally, the inspection coverage capability, flight safety, and engineering deployment availability of the proposed strategy were comprehensively evaluated based on indicators such as damage target detection rate, beam bottom coverage, pier coverage, flight time, minimum safe distance, and number of collisions. The flight control closed-loop verification results are shown in Table 4.

[0146] Table 4 Overall Results of Bridge Inspections at Actual Proportion

[0147] As shown in Table 4, the UAV was able to complete the detection of all damaged targets in the flight control closed-loop system, and no collision occurred during the flight. The bottom surface of the beam and the pier surface both achieved high coverage, indicating that the speed command access, pose feedback, Gaussian rendering closed loop, yaw point strategy and dual altitude inspection strategy in step six can support the autonomous inspection verification in a real-scale bridge scenario.

[0148] This invention constructs a 3D Gaussian training scenario for bridges, combining Gaussian new perspective rendering microservices, Euclidean symbolic distance field security constraints, multimodal observation space, multi-objective reward function, PPO model-free reinforcement learning method, SHAC differentiable policy optimization method, and PX4 / Gazebo / MAVROS flight control closed-loop feedback mechanism to achieve autonomous coverage inspection, damage target observation, path decision training, and flight control closed-loop verification of UAVs in complex bridge structure environments.

[0149] Meanwhile, for those skilled in the art, based on the ideas of this invention, the specific source and format of the bridge 3D model, the construction method of the 3D Gaussian scene, the Gaussian primitive initialization method, the rendering image resolution, the visual observation type, the ESDF distance field resolution, the number of coverage sampling points, the coverage determination threshold, the semantic category division method, the weight coefficients of each item in the reward function, the policy network structure, the form of the visual encoder, the form of the state encoder, the number of layers of the Actor-Critic network, the number of neurons per layer, the PPO pruning threshold, the SHAC short-time domain expansion length, the range of domain randomization parameters, the flight control interface type, the speed command issuance method, the simulation platform configuration, and the evaluation index settings can all be changed and adjusted according to the actual bridge structure type, UAV platform performance, sensor configuration, and inspection task requirements.

[0150] The present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the bridge unmanned aerial vehicle autonomous inspection method based on Gaussian scene perception and reinforcement learning.

[0151] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the bridge unmanned aerial vehicle autonomous inspection method based on Gaussian scene perception and reinforcement learning.

[0152] The memory in this application embodiment can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the methods described in this invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0153] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0154] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0155] It should be noted that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as execution by a hardware decoding processor, or as a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0156] The above provides a detailed description of the bridge unmanned aerial vehicle (UAV) autonomous inspection method based on Gaussian scene perception and reinforcement learning proposed in this invention. Specific examples are used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A bridge unmanned aerial vehicle (UAV) autonomous inspection method based on Gaussian scene perception and reinforcement learning, characterized in that, The method includes: Step 1: Construct a 3D Gaussian training scene for UAV autonomous inspection to represent the texture, geometry, depth, and semantic information of the bridge structure; Step 2: Establish a Gaussian rendering and geometric safety constraint module to generate UAV visual observations in real time and complete safe distance queries, collision judgments, and coverage statistics; Step 3: Construct a reinforcement learning task model for UAV autonomous inspection, taking visual observation, task status, local geometric information and flight control status as inputs, and speed commands as action outputs; Step 4: Design a multi-objective reward function for coverage inspection to uniformly evaluate coverage gain, information gain, pose alignment, semantic priority, security risks, and invalid behaviors; Step 5: Train the UAV autonomous inspection policy network using the near-end policy optimization method and the short-time domain differentiable policy optimization method, and improve the policy robustness through the domain randomization mechanism; Step Six: Integrate the trained UAV autonomous inspection strategy into the flight control closed-loop system to achieve closed-loop verification, flight execution, and engineering deployment of the inspection strategy.

2. The method according to claim 1, characterized in that, Step one specifically includes: Step 11: Obtain multi-source scene data of the inspection object; Steps 1 and 2: Establish bridge scene pose constraints and initialize 3D Gaussian primitives. Use the precise camera pose output by the virtual camera in the Gazebo simulation environment as the pose input for training the 3D Gaussian scene. Then, initialize the Gaussian primitive positions by sampling point clouds from the bridge mesh model, reconstructing point clouds from multiple views, or back-projecting point clouds from the simulation depth. Step 13: Jointly optimize the 3D Gaussian scene parameters of the bridge. Utilize multi-view RGB image photometric reconstruction loss, structural similarity loss, simulation depth supervision term, geometric anchoring constraints, and depth correlation constraints to jointly optimize the Gaussian meta-parameters, so that the rendered image is consistent with the bridge simulation observation in terms of color, structure, depth, and geometric distribution. Step 14: Construct additional scenario information for bridge inspection tasks.

3. The method according to claim 1, characterized in that, Step two specifically includes: Step 21: Before reinforcement learning training begins, load the 3D Gaussian scene model into an independent rendering process, keep the Gaussian model resident in video memory, and provide on-demand rendering services to the outside world through a lightweight interface. Step 22: At each time step, the physics simulator or flight control system outputs the UAV's six-degree-of-freedom pose in the global coordinate system, based on the camera's extrinsic parameters. The drone system pose is converted into a camera pose, and the camera pose is then passed to the Gaussian rendering backend. Steps 2 and 3: The Gaussian rendering backend generates an RGB image, depth map, and semantic map from the current viewpoint based on Gaussian primitive projection, spherical harmonic color representation, and transparency accumulation mechanism; Step 24: Convert the Gaussian scene or its point cloud proxy into an Euclidean symbolic distance field (ESDF); Step 25: Construct an ESDF voxel mesh at a preset resolution, and obtain the distance between the UAV's current position and the nearest obstacle through trilinear interpolation during training; Step 26: Sample target points from the ESDF occupied surface, bridge mesh surface, or Gaussian point cloud proxy. Each target sampling point contains spatial coordinates, surface normal vector, minimum observation cosine threshold, and optional semantic category. Step 27: Establish an interface between Gaussian new perspective rendering, ESDF safe distance query, coverage sampling point statistics, and reinforcement learning step() process, so that visual observation, geometric safety information, and coverage reward can be updated synchronously at each time step.

4. The method according to claim 1, characterized in that, Step three specifically includes: Step 31: Model the UAV autonomous inspection task as a Markov decision process. At each time step, the agent selects an action based on the current observation, the environment updates the UAV state based on physical constraints, and returns the new observation and reward. Step 32: Construct a multimodal observation space composed of visual realism, task progress, and local security; Step 33: Define the action space as a four-dimensional continuous velocity command and use the velocity command as the action output so that the strategy can adapt to speed limits, inertial response, control delay and underlying flight control constraints during the training phase.

5. The method according to claim 1, characterized in that, Step four specifically includes: Step 41: Discretize the target structure surface into a set of covering sampling points; Step 42: For any covered sampling point, determine whether the point has been effectively observed based on the UAV camera center position, camera optical axis direction, target point distance, camera field of view angle, and target point surface normal vector; Step 43: Use a persistent overlay mask to record the overlay status. Initially, all overlay points are in an unobserved state. Update the overlay mask at each time step based on the visibility results. Step 44: Construct a frontier exploration score to measure the extent to which the current camera is pointing towards uncovered areas; Steps four and five: Constructing pose alignment rewards; Step 46: Construct semantic priority rewards; Step 47: Construct safety penalties and collision penalties; Step 48: Combine the coverage reward, information gain reward, pose alignment reward, semantic priority reward, stagnation penalty, boundary penalty, safety penalty, and collision penalty into a total reward function.

6. The method according to claim 1, characterized in that, Step five specifically includes: Step 51: Construct a multimodal Actor-Critic policy network; Step 52: Use SHAC to perform differentiable pre-training on the Actor-Critic policy network; Step 53: Use the PPO method to perform high-fidelity fine optimization on the SHAC pre-trained policy network; Step 54: Employ a domain randomization mechanism to improve policy robustness.

7. The method according to claim 1, characterized in that, Step six specifically includes: Step 61: Convert the continuous actions output by the policy network into flight control speed commands; Step 62: Establish a closed loop for pose feedback, Gaussian rendering, and time synchronization; Step 63: Conduct strategy verification and engineering usability evaluation in a real-scale bridge simulation environment.

8. The method according to claim 6, characterized in that, The multimodal Actor-Critic policy network includes a visual encoding branch, a state encoding branch, a shared feature fusion module, an Actor output head, and a Critic output head. The visual encoding branch processes RGB images, depth maps, or semantic maps obtained from the rendering of a 3D Gaussian scene of the bridge, extracting bridge structural texture, component boundaries, local occlusion, damage appearance, and spatial geometric features. The state encoding branch processes low-dimensional task states, including coverage, recent coverage gain, frontier exploration score, safe distance, attitude alignment score, semantic target completion, remaining range, or flight control feedback state. These two types of features are concatenated and input into a shared multilayer perceptron in the shared feature fusion module for fusion, resulting in a joint feature representation for policy decision-making and value estimation. The Actor output head outputs continuous action distribution parameters based on the joint feature representation, used to generate UAV 3D linear velocity and yaw rate commands. The Critic output head outputs a state value estimate based on the joint feature representation, used to calculate the advantage function, value function loss, and policy update objective.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-8.

10. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-8.

Images