Pipeline inspection unmanned aerial vehicle navigation method and system based on reinforcement learning

Abstract

The application belongs to the technical field of unmanned aerial vehicle navigation. A pipeline inspection unmanned aerial vehicle navigation method and system based on reinforcement learning are provided. By collecting inertial measurement, visual inertial odometry, cross array ranging and electronic governor feedback data, a state vector is constructed and high-frequency motion prediction and measurement update are realized through extended Kalman filtering to output high-precision unmanned aerial vehicle body state. The best anti-interference height is solved by combining the pipeline wind disturbance and rotor airflow disturbance model, and the external disturbance torque is decoupled. A dual-frequency reinforcement learning state is constructed and control instructions are output, and safe speed is obtained through safety shield filtering. With safe speed as the target, the motor speed is directly output through the reinforcement learning network to control flight, and the three-dimensional topological mapping of pipeline defects is completed synchronously. The application can realize high-precision navigation, active disturbance rejection and safe inspection in the pipeline, and improve the flight stability and operation reliability in complex aerodynamic environment.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) navigation technology, specifically to a pipeline-based UAV navigation method and system based on reinforcement learning. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Miniature unmanned aerial vehicles (UAVs), with their three-dimensional maneuverability, are widely used in non-contact inspections of confined spaces such as petrochemical pipelines, urban underground utility tunnels, nuclear power plant ventilation shafts, and HVAC ducts. They can traverse vertical pipe sections and avoid obstacles within the pipes, overcoming the limitations of traditional pipeline robots. In narrow, enclosed pipe environments, the downwash airflow from the UAV rotor violently impacts the pipe walls, top, and bottom, creating vortices and backflows. This induces nonlinear aerodynamic disturbances such as ground effect, ceiling effect, wall effect, and corner effect. These fluid-structure interaction effects have become a key research focus in the control of pipeline inspection UAVs, and multimodal perception and adaptive control technologies have become major development directions in the industry.

[0004] Existing pipeline inspection drones generally adopt a cascaded control architecture, relying on a fixed-parameter feedback controller at the bottom layer. This architecture has significant drawbacks in the highly disturbed environment of pipelines, making it impossible to achieve stable and reliable autonomous navigation and flight control. Because the controller struggles to adaptively match the dynamically changing aerodynamic disturbances within the pipeline, and the delay caused by cascaded calculations leads to system response lag, and because it cannot actively utilize pipeline geometry and flow field coupling information for feedforward disturbance rejection, the drone struggles to accurately estimate its own state and external disturbance torques. Furthermore, it cannot achieve rapid and stable control by directly driving the motors end-to-end, thus failing to accurately detect and locate pipeline defects while ensuring safety. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a pipeline inspection UAV navigation method and system based on reinforcement learning, which achieves high-precision and high-frequency continuous state calculation within a confined space with extremely low computational and hardware load.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a pipeline inspection UAV navigation method based on reinforcement learning.

[0007] A pipeline-based unmanned aerial vehicle (UAV) navigation method based on reinforcement learning includes the following process: Acquire inertial measurement data, visual inertial odometry data, cross array ranging data, and electronic speed control feedback data of the UAV to construct continuous-time state vectors and error state vectors; Based on extended Kalman filtering, high-frequency motion prediction is performed using inertial measurement data by utilizing continuous-time state vectors and error state vectors. Visual inertial odometry data and cross array ranging data are asynchronously fused to complete measurement updates and state injection, outputting high-precision UAV body angular velocity, global position, body attitude and body linear velocity. Based on the duct wind disturbance model and the rotor airflow disturbance model, using high-precision body angular velocity, global position and ESC feedback data, the comprehensive airflow disturbance torque is calculated and the optimal anti-disturbance altitude is solved. The external disturbance torque is obtained by decoupling the actual total torque currently borne by the UAV from the theoretical output torque of the motor. Using the optimal anti-disturbance altitude, external disturbance torque, global position, body linear velocity and cross array ranging data, a navigation reinforcement learning state and a control reinforcement learning state are constructed. Based on the navigation reinforcement learning state and the control reinforcement learning state, control commands are output through continuous action mapping. A safe speed command is obtained by filtering through a safety shield. With safe speed as the tracking target, and combining the control reinforcement learning state, external disturbance torque, body attitude, body angular velocity and body linear velocity, the reinforcement learning network directly outputs the motor speed to control the UAV flight. Simultaneously, based on the global position and body attitude, it completes the three-dimensional absolute topology mapping of the pipeline defect and enters the next control loop.

[0008] In one implementation of the first aspect of the present invention, performing high-frequency motion prediction includes: The extrapolated value of the body attitude at the next moment is calculated by integrating the quaternion differential equation using the body angular velocity after zero bias calibration. The specific force under the machine system is projected onto the global coordinate system through the rotation matrix of the machine attitude. After canceling the gravity vector, the linear velocity of the machine is obtained by the first integration and the global position is obtained by the second integration. The state covariance matrix is ​​predicted by using the state transition Jacobian matrix and the inertial measurement process noise matrix. The body attitude, body linear velocity, global position and state covariance matrix are all used for subsequent measurement updates and state injection.

[0009] In one implementation of the first aspect of the present invention, performing measurement update and state injection includes: The velocity residual between the predicted linear velocity and the observed linear velocity of the organism is calculated using visual inertial odometry data. Using cross array ranging data, the ray projection method combined with the geometric equation of the cylindrical surface of the pipe is used to solve for the theoretical ranging value, and the distance residual between the theoretical ranging value and the measured ranging value is calculated. The velocity and distance residuals are mapped to the error states through the corresponding Jacobian matrices. The Kalman gain is then calculated to complete state compensation and covariance update. The resulting global position, body attitude, body angular velocity, and body linear velocity are used for subsequent torque calculation and reinforcement learning state construction.

[0010] In one implementation of the first aspect of the present invention, solving for the optimal anti-interference height includes: Establish a pipeline wind disturbance model corresponding to fully developed pipeline flow, and a rotor ground effect and ceiling effect disturbance model based on the image method; The disturbance moment generated by the duct wind disturbance model and the disturbance moment generated by the rotor airflow disturbance model are superimposed to obtain the comprehensive airflow disturbance moment. The optimization objective is to minimize the sum of squares of the comprehensive airflow disturbance moment. The optimal anti-disturbance height is solved by setting the derivative to zero or by using the gradient descent method to find the minimum airflow disturbance in the duct. The optimal anti-disturbance height is used for reinforcement learning navigation state construction and altitude tracking constraint calculation.

[0011] In one implementation of the first aspect of the present invention, decoupling to obtain the external disturbance torque includes: The angular acceleration is obtained by numerically differentiating and low-pass filtering the body angular velocity output by the extended Kalman filter, and then the total torque actually experienced by the UAV is calculated by combining the body diagonal inertia tensor. The theoretical output torque of the motor is calculated based on the motor speed, lift coefficient, torque coefficient and lever arm length in the ESC feedback data. The difference between the actual total torque currently experienced by the drone and the theoretical output torque of the motor is used to decouple and obtain the external disturbance torque of the unmodeled airflow in the duct. The external disturbance torque is used for reinforcement learning state construction and speed control calculation.

[0012] In one implementation of the first aspect of the present invention, constructing a navigation reinforcement learning state and a control reinforcement learning state and outputting control commands includes: The navigation reinforcement learning state is constructed by fusing cross-array ranging data, body linear velocity, optimal anti-disturbance altitude, global position, and the previous cycle velocity command. The system's speed error, body attitude, body angular velocity, external disturbance torque, and motor speed at the previous moment are used to construct the control reinforcement learning state. The navigation reinforcement learning state is input into the navigation network, and a three-dimensional velocity command is output through continuous motion mapping. The control reinforcement learning state is input into the control network, and a motor speed command is output through continuous motion mapping. The three-dimensional velocity command is used for safety shield filtering, and the motor speed command is used for direct control of the UAV.

[0013] As a further limitation of the first aspect of the present invention, the execution of the security shield filtering to obtain the security speed command includes: Based on the velocity obstacle theory, the inner wall of the pipe is modeled as a static obstacle and a collision cone is constructed using global position and cross array ranging data. When the three-dimensional velocity command output by the navigation network falls into the collision cone, the quadratic programming solver is activated. With the goal of minimizing the distance to the dangerous command, the safe speed is calculated within the safe speed range and used as the tracking target of the control network. The safe speed is used for the velocity error calculation of the control network and the construction of the control reinforcement learning state.

[0014] In one implementation of the first aspect of the present invention, completing the three-dimensional absolute topology mapping of pipeline defects includes: Extract the global position and body attitude from the extended Kalman filter output at a fixed spatial frequency; The relative position of the pipe defect in the camera coordinate system is projected through a rigid body transformation of the coordinate system, and the global absolute position of the defect is obtained by combining the global position and the body attitude. The global absolute position of the defect is fused with the ranging point cloud and timestamp to output a 3D topology defect log of the pipeline network and trigger UAV hovering recording. The global absolute position of the defect is used for navigation model switching and hovering control.

[0015] In one implementation of the first aspect of the present invention, controlling the drone to fly and enter the next control cycle includes: The control reinforcement learning state and safe speed are input into the reinforcement learning control network, which outputs the corresponding motor speed command. This command is then sent to the ESC via a low-latency digital protocol to drive the four rotors to adjust their speed. The interaction between rotor thrust and torque and airflow within the duct generates new displacements and disturbances. The control clock then enters the next cycle and returns to execute the multimodal sensor data acquisition step. The newly generated displacements and disturbances are used for the inertial measurement and cross array ranging data acquisition in the next cycle.

[0016] Secondly, the present invention provides a pipeline inspection drone navigation system based on reinforcement learning.

[0017] A pipeline inspection UAV navigation system based on reinforcement learning, comprising: The data acquisition unit is configured to acquire inertial measurement data, visual inertial odometry data, cross array ranging data, and electronic speed control feedback data of the UAV, and construct a continuous-time state vector and an error state vector. The state calculation unit is configured to: perform high-frequency motion prediction based on extended Kalman filtering, using continuous-time state vectors and error state vectors, asynchronously fuse visual inertial odometry data and cross array ranging data to complete measurement updates and state injection, and output high-precision UAV body angular velocity, global position, body attitude and body linear velocity. The torque calculation unit is configured to: calculate the comprehensive airflow disturbance torque and solve the optimal anti-disturbance altitude based on the duct wind disturbance model and the rotor airflow disturbance model, using high-precision body angular velocity, global position and ESC feedback data; and obtain the external disturbance torque by decoupling the actual total torque currently borne by the UAV from the theoretical output torque of the motor. The strategy construction unit is configured to: construct navigation reinforcement learning state and control reinforcement learning state using the optimal anti-disturbance altitude, external disturbance torque, global position, body linear velocity and cross array ranging data; output control commands based on navigation reinforcement learning state and control reinforcement learning state through continuous action mapping; and obtain safe speed commands through safety shield filtering. The flight control unit is configured to: use a safe speed as the tracking target, combine the control reinforcement learning state, external disturbance torque, body attitude, body angular velocity and body linear velocity, and directly output motor speed to control the flight of the UAV by the reinforcement learning network. Simultaneously, it completes the three-dimensional absolute topology mapping of the pipeline defect based on the global position and body attitude and enters the next control loop.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This invention innovatively proposes a pipeline inspection UAV navigation method based on reinforcement learning, which effectively solves the problems of traditional control architectures being unable to adaptively suppress aerodynamic disturbances, having lag response, and being difficult to accurately locate and safely navigate within the confined space of pipelines. First, the fusion of multi-source data acquisition and extended Kalman filtering enables high-precision output of the drone's status in dark pipeline environments without GPS, providing a reliable state foundation for subsequent control. Second, by solving for the optimal anti-disturbance altitude based on an aerodynamic model and decoupling external disturbance torque in real time, the influence of turbulence can be avoided from the physical source, while accurately quantifying nonlinear airflow interference within the pipeline, providing a basis for active anti-disturbance. Third, by constructing a dual-frequency reinforcement learning state and outputting control commands, combined with a safety shield filtering mechanism, traditional cascaded control loops can be bypassed, achieving ultra-low latency end-to-end motor speed control, quickly offsetting airflow disturbances and eliminating collision risks. Finally, the drone is directly driven to fly at a safe speed, while simultaneously completing three-dimensional topology mapping of pipeline defects. Under the premise of ensuring flight stability and safety, the integrated execution of autonomous navigation and defect detection within the pipeline is achieved, comprehensively improving the drone's anti-disturbance capability, control response speed, and inspection operation reliability in narrow pipeline environments, ensuring continuous and stable operation in complex aerodynamic environments.

[0019] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0021] Figure 1 A schematic diagram of the framework of a pipeline inspection UAV navigation method based on reinforcement learning, provided as an exemplary embodiment of the present invention; Figure 2 A schematic diagram of a process for predicting the state of a drone based on extended EKF, provided as an exemplary embodiment of the present invention; Figure 3 A schematic diagram illustrating the real-time estimation of optimal flight altitude and airflow disturbance torque as provided in an exemplary embodiment of the present invention; Figure 4 A flowchart illustrating a reinforcement learning-based inspection strategy provided as an exemplary embodiment of the present invention; Figure 5 This is a schematic diagram of a reinforcement learning-based unmanned aerial vehicle (UAV) navigation system for pipeline inspection, provided as an exemplary embodiment of the present invention. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] Existing micro unmanned aerial vehicle control systems, when facing narrow pipe environments, are unable to effectively overcome the strong unsteady aerodynamic disturbances caused by ground effect, ceiling effect, wall effect and corner effect due to the inherent limitations of cascaded passive feedback architecture and fixed parameter control law. They also suffer from poor robustness, slow response and lack of active utilization of space environment and flow field coupling information.

[0025] To overcome the challenges faced by micro-drones in spatial positioning within confined, narrow, and poorly lit pipe environments, and their susceptibility to crashes due to complex airflow vortices, this invention proposes an ultra-lightweight integrated sensing and control solution, such as... Figure 1As shown, this scheme employs a multimodal sensing array composed of a cross-shaped Time-of-Flight (ToF) sensor, a visual inertial odometry (VIO) system, and an inertial measurement unit (IMU) to acquire multi-source environmental and self-state data with extremely low computational and hardware load. Subsequently, the system utilizes an extended Kalman filter (EKF) to deeply fuse the aforementioned sensor data, accurately calculating the high-frequency motion state in complex dark environments. Finally, this high-precision state, along with the real-time estimated airflow disturbance torque, is used as the input to the underlying reinforcement learning neural network, thereby overcoming the latency of traditional controllers and outputting ESC data. This enables direct, ultra-low-latency end-to-end active disturbance rejection control of the four motor speeds, dynamically offsetting complex airflow disturbances within the pipeline and ensuring the safety and stability of the inspection mission.

[0026] Specifically, the reinforcement learning-based pipeline inspection UAV navigation method of the present invention includes the following processes: S1: Multimodal sensor data acquisition and preprocessing.

[0027] The system maintains a high-frequency real-time clock (set to). In each control cycle Internally, the following multi-source hardware data are simultaneously collected and preprocessed: Inertial Measurement Unit (IMU): Frequency acquisition of three-axis acceleration of the machine body With triaxial angular velocity The filter is passed through a second-order Butterworth low-pass filter (with a cutoff frequency set to...). Filtering out high-frequency vibration noise from the rotor blades, the purified angular velocity of the airframe is obtained. .

[0028] Airborne Visual Inertial Odometry (VIO): Utilizes a global shutter miniature camera combined with an independent IMU to... Frequency acquisition of the translational velocity observations of the body in the initial local coordinate system and relative position increment .

[0029] Cross-array ToF sensor: Frequency polling retrieves the absolute distance scalars from the wall in the forward, left, right, up, and down directions. .

[0030] Electronically regulated feedback (ESC Telemetry): Frequency closed-loop reading reads the actual speed feedback vectors of the four brushless motors at the previous moment. .

[0031] S2: Drone state prediction based on extended EKF, such as Figure 2 As shown.

[0032] S2.1: Definition of the state-space model.

[0033] The system first constructs a continuous-time state vector. This is used to characterize the complete motion features of a drone in three-dimensional space. (1); in, and These represent the three-dimensional position vector and the three-dimensional linear velocity vector of the UAV in the global coordinate system, respectively. This represents the body posture using quaternions to avoid the gimbal lock problem in the traditional Euler angle representation. and These represent the random time-varying zero bias of the accelerometer and gyroscope in the inertial measurement unit (IMU), respectively. This zero bias drifts over time and needs to be calibrated online in real time during the filtering process.

[0034] Define the error state vector as a 15-dimensional tensor. : (2); in, and These represent the three-dimensional position error and three-dimensional linear velocity error of the UAV in the global coordinate system, respectively. It represents the three-dimensional attitude error angle, used to approximate the error of the quaternion by assuming a small angle; and These represent the random time-varying zero bias errors of the accelerometer and gyroscope in the inertial measurement unit (IMU), respectively.

[0035] S2.2: High-frequency motion prediction stage.

[0036] The system uses the high frequency sampled by the IMU as a reference clock to run nonlinear kinematic prediction equations, changing the state from Time extrapolation to time: Attitude dynamics evolution: utilizing the body angular rate after zero bias calibration The attitude extrapolation value at the next time step is calculated by integrating the quaternion differential equation. : (3); in, This represents the control cycle time interval; This represents the quaternion exponent mapping.

[0037] Linear motion evolution: Comparison under machine system After rotation matrix Projected onto the global coordinate system, and the gravity vector is canceled out. Then, the velocity is obtained by one integration, and the position is obtained by a second integration. (4); (5); in, This represents the extrapolated global position value at the next moment; This represents the three-dimensional linear velocity vector at the next moment.

[0038] Simultaneously, the system synchronously predicts the state covariance matrix. This matrix characterizes the uncertainty of the current state estimate. The prediction process is achieved through the state transition Jacobian matrix. With IMU process noise array : (6); in: This corresponds to the error state. The discretized form of the continuous-time Jacobian matrix. It is the process noise covariance matrix of the IMU; This represents the covariance matrix of the predicted state at the next moment.

[0039] S2.3: Asynchronous Measurement Updates and State Injection.

[0040] When external observation data arrives, the system enters an update cycle. To ensure versatility, this invention can abstract the updates of VIO and ToF into a unified process: When the airborne visual inertial odometry (VIO) is in Output translational velocity observations in the world coordinate system at all times. At that time, due to the presence of Gaussian white noise in VIO itself The actual observation equation can be expressed as: (7); in, Represents VIO velocity observations; Represents the actual speed at the current moment.

[0041] The system directly extracts the predicted speed under the current state. This is compared with the high-precision world coordinate system velocity output by VIO. Subtract to obtain the three-dimensional velocity residual. : (8); To map the 3D velocity residual to the 15D error state Above, construct the measurement Jacobian matrix. Since velocity observation is only related to the velocity components in the state vector. For direct linear dependence, the Jacobian matrix is ​​expressed in a simplified block matrix form. : (9); in, Represents a 3×3 zero matrix. It represents a 3×3 identity matrix.

[0042] The system utilizes the currently predicted 3D pose By combining the ray casting method with the known pipe cross-section equations, the theoretically measurable distance vector at that moment can be calculated. .

[0043] Known ToF sensor The installation translation vector in the UAV body coordinate system is: The unit vector of the beam emission direction is Based on the current nominal position of EKF and attitude rotation matrix The starting point of the ray and direction Transform to global coordinate system: Ray origin : (10); in, The X-axis, Y-axis, and Z-axis coordinates represent the starting point.

[0044] Ray direction: (11); These represent the XYZ three-axis components of the ray direction.

[0045] Assuming the inspected pipe is a standard circular pipe, its central axis coincides with the X-axis of the global coordinate system, and the pipe radius is... The geometric equation of the inner wall of the pipe is: (12).

[0046] in, and The coordinates of the cross section in the global coordinate system are x (the pipe axis is x).

[0047] The coordinates of any point on the ray are ,in, This is the distance expected to be measured by ToF. Substituting the ray into the equation of the cylindrical surface: (13); After unfolding, we can obtain information about The quadratic equation of : (14); in: (15); (16); (17); Solving this quadratic equation (taking the positive real roots) yields the theoretically expected distance that Time-of-Flight (ToF) should measure at the current pose: (18); Compare it with the distance measurement actually returned by the hardware. Compare and obtain the scalar distance residual. : (19); Due to the predicted distance Location of the drone and posture It exhibits a highly non-linear relationship and constructs... Jacobian matrix of dimension : (20); in, It represents a 1×3 zero matrix.

[0048] Partial derivative with respect to position This determines how ToF distance errors correct for the drone's lateral and altitude deviations.

[0049] Partial derivative with respect to attitude It determines how ToF errors correct the yaw and pitch errors of the UAV.

[0050] Based on which sensor triggered the update, retrieve the corresponding Jacobian matrix and measurement noise covariance matrix, and solve for the Kalman gain. : (twenty one); in, This represents the prior state error covariance matrix; The collective observation Jacobian matrix is ​​specifically... or It is used to map the low-dimensional residuals of the observation space to a 15-dimensional state space; This represents the transpose of the observation Jacobian matrix; This refers to the general term "measurement noise covariance matrix," specifically indicating... or This characterizes the physical noise level of the sensor's own hardware.

[0051] Both the 3D VIO velocity residual and the 6D ToF array distance residual need to be precisely assigned to the UAV's 15-dimensional complex state system through matrix multiplication: (twenty two); in, This refers to the general term "observation residuals," specifically referring to... or .

[0052] The system inversely maps low-dimensional observational errors to underlying real physical errors. .

[0053] After absorbing effective information from external sensors, the system has a more confident understanding of the current state of the drone. Therefore, according to the formula: (twenty three); in, The prior state error covariance matrix represents the degree of uncertainty of the current system's internal state estimation. It is the identity matrix; The updated posterior state error covariance matrix reduces system uncertainty after fusing observation data.

[0054] The system shrinks the state covariance matrix, which mathematically reduces the estimation uncertainty of the entire system.

[0055] Then, the obtained error amount Compensation to continuous-time state middle.

[0056] Position, velocity, zero bias (additive injection): (twenty four); (25); (26); (27); in, This is the updated position; For the updated speed; The updated accelerometer has zero bias. This ensures zero bias in the updated gyroscope.

[0057] Attitude (multiplicative injection, quaternion update): Using attitude error angle Constructing tiny rotation quaternions Then multiply it by the nominal attitude: (28); After the injection is complete, the error has been absorbed by the nominal state, so the error state vector is reset to zero to prepare for the next IMU integration.

[0058] (29); S3: Real-time estimation of optimal flight altitude and airflow disturbance moment, such as Figure 3 As shown.

[0059] This module runs on High frequency (HF) involves two core sub-processes: first, predicting the airflow equilibrium surface within the duct based on an aerodynamic prior model to output the optimal cruising altitude; and second, converting the duct wall effect and backflow quantification into observable real-time torque residuals.

[0060] S3.1: Establishment of the duct wind disturbance model and the rotor airflow disturbance model.

[0061] Within a confined space, the aerodynamic disturbance torque experienced by the UAV mainly originates from two parts: the wind field distribution inherent to the duct environment, and the backflow generated by the UAV's own rotor downwash impacting the duct wall. The system establishes the following one-dimensional height (Z-axis) dependent prior model: Pipe Wind Profile Model: Assume the flow in the pipe is a fully developed pipe flow (Poisson flow), and the ambient wind speed exhibits a parabolic distribution across its cross-section. Let the inner diameter of the pipe be... The height from the bottom of the pipe is The maximum wind speed at the center of the pipeline is The roll / pitch disturbance moment function induced by wind speed gradient shear stress is... It can be modeled as: (30); in, This is a constant related to the windward area of ​​the UAV fuselage and the aerodynamic drag coefficient.

[0062] Rotor Recirculation Model for Unmanned Aerial Vehicles: The downwash airflow generated by the high-speed rotation of the rotor produces a strong ground effect (GE) and ceiling effect (CE) when it touches the bottom and top of the pipe, inducing asymmetric backflow disturbance torque. Establish an inverse squared distance decay model based on the method of images: (31); in, This represents the current average hovering thrust. and These are the backflow interference coefficients for the ground effect and ceiling effect, respectively, calibrated by fluid dynamics (CFD).

[0063] S3.2: Optimal height for predicting the minimum airflow disturbance moment ( ).

[0064] The system superimposes the two models to obtain the arbitrary height inside the pipe. The comprehensive predictive airflow disturbance moment functional : (32); Before generating the vertical control law, the high-level navigation planning module performs a process of... Differentiate and let Alternatively, the gradient descent method can be used to solve in real time online for the optimal anti-disturbance equilibrium height where airflow disturbance is minimized and the effects of the upper and lower airflows cancel each other out. : (33); The solution It is directly transmitted to the upper layer as the locking reference for the Z-axis vertical control command, thereby realizing the strategy of "avoiding the strongest turbulence from the physical source".

[0065] S3.3: Calculate the current total torque of the system ( ).

[0066] For transient nonlinear turbulence that the model cannot cover, the system activates a real-time observer. This provides high-precision angular velocities from the EKF output. The actual angular acceleration is obtained by performing numerical differentiation and low-pass filtering. Based on the body's diagonal inertia tensor The total torque actually borne by the current system is: (34); S3.4: Calculate the theoretical output torque of the motor ( ) Based on the actual speed feedback from the electronic speed controller (ESC) Lift coefficient Torque coefficient and lever arm length Theoretically, the output torque issued by the drone controller is: (35); S3.5: Decoupling unmodeled external disturbance torque ( ).

[0067] The system performs algebraic subtraction in real time to extract the torque generated by its own motor from the true total torque. The remaining residual is the net disturbance torque exerted on the drone by the complex turbulence in the pipe at the current instant. (36); The residual The "lateral suction caused by the wall effect" or "sudden torque caused by local eddies" was quantified with extremely high precision, and then fed directly to the underlying reinforcement learning network as a core state variable, triggering a microsecond-level motor compensation response.

[0068] S4: Inspection strategies based on reinforcement learning, such as Figure 4 As shown.

[0069] The system as a whole exhibits a dual-frequency decoupled operation mechanism: with The frequency-operating navigation network (Actor 1) is responsible for macroscopic obstacle avoidance and path planning in three-dimensional space; The frequency-operated control network (Actor 2) is responsible for microscopic aerodynamic disturbance suppression and target command tracking. The two are rigidly coupled mathematically and logically through the "velocity command flow" and the "safety shield".

[0070] The following are the specific implementation steps of this algorithm: S4.1: Joint state space extraction and dimensionality reduction construction.

[0071] The system extracts scalar data from each ToF sensor and encapsulates it into a spatial ray range vector. : (37); in, These represent the distance measurement data for the front, left, right, bottom, and top, respectively.

[0072] Then the system will By integrating with the internal state of EKF, a navigation MDP state input tensor is constructed. : (38); in, The absolute linear velocity of the machine body estimated by EKF; The optimal wind-resistant balance height output by the front module; This indicates the absolute location of the pipeline rupture, used for hovering and recording the extent of the rupture. The desired velocity from the previous control cycle is used to provide the network with first-order temporal momentum, ensuring the temporal smoothness of the action output.

[0073] The control network's role is to precisely absorb nonlinear aerodynamic residuals. The system receives the absolute safe speed from the navigation network, filtered through a safety shield. Build the underlying state : (39); in, This represents the three-dimensional velocity tracking error mapped to the body coordinate system; For attitude quaternions; Angular velocity; For real-time calculation of the residuals of the three-axis airflow disturbance torque; The actual rotational speed of the four motors at the previous moment is given. The introduction of this term enables the network to implicitly learn and compensate for the physical execution lag of the brushless motor. This is a safe speed command.

[0074] S4.2: Boundary constraint strategy inference based on Beta distribution.

[0075] Continuous mapping of desired velocity: Navigation Actor 1 performs forward computation using a multilayer perceptron (MLP), and instead of directly outputting velocity values, it outputs two strictly positive parameters that determine the shape of the Beta distribution. .

[0076] During execution, the system takes the mathematical mean of this distribution as the normalized action benchmark. And linearly inversely map it into a three-dimensional velocity command in the physical coordinate system. : (40); This formula ensures that the instructions issued by the neural network will always converge absolutely to the... Within the physical security envelope, This represents the upper limit of the speed amplitude.

[0077] High-frequency mapping of motor speed: The Actor 2 controller also outputs Beta distribution parameters, extracting a normalized command vector containing the four rotor channels. By using the inverse normalization formula, it is directly mapped to a physical speed recognizable by the electronic speed controller (ESC): (41); in, This is the maximum speed of the motor; This is the minimum speed of the motor.

[0078] S4.3: Speed-based obstacle (VO) safety shield.

[0079] The "black box" nature of deep neural networks can lead to extremely low-probability, potentially fatal violations (such as a drone crashing directly into a pipe wall). To meet the requirements of industrial inspection... Due to the mandatory safety requirements, this invention forcibly embeds a non-learning analytical geometry called "Safety Shield" between Actor 1 and Actor 2.

[0080] Collision cone construction: The system utilizes the current EKF position and the output of the ToF array. The inner wall of the pipe is modeled as a group of static geometric obstacles. Based on the Velocity Obstacle (VO) theory, a set of projective cones pointing from the robot's current position to all potential collision points is calculated.

[0081] Online Quadratic Programming (QP) Projection: If Actor 1 experiences a sudden hallucination, what is the expected output velocity? Falling within the collision cone (i.e., judging the future) (A collision is inevitable within seconds), and the safety shield will forcibly intercept the command.

[0082] The system activates the online quadratic programming solver in real time to find the distance from the original dangerous command. Recent Absolutely Safe Speed : (42); in, This ensures that the solution lies outside the half-plane corresponding to the collision. This satisfies mobility constraints.

[0083] The solution obtained Will replace This unalterable final goal is then sent to the underlying Actor 2 for execution. This mechanism provides a safety net for the neural network, eliminating the risk of system crashes.

[0084] S4.4: Reward function.

[0085] The macro-level reward mechanism for navigation (Critic 1): The system completely eliminates the need for manual, repeated tuning of proportional-derivative (PD) rule parameters and designs a four-dimensional incentive reward system: (43); in, , , and All are weights.

[0086] Kinetic Energy and Guiding Rewards ( Kinetic energy and guidance rewards guide the drone to overcome resistance and move forward along the virtual corridor.

[0087] Logarithmic static security penalty ( ): Extract the ToF ray distance and apply a logarithmic function penalty: (44); When extremely close to an obstacle ( As the negative penalty tends to infinity, it forces the network to generate a subconscious tendency to actively move away from the pipe wall.

[0088] Smoothness penalty ( The calculation formula is the norm of the difference between two consecutive frame speed commands, which is used to eliminate motion jitter.

[0089] (45); Height lock ( ): Regarding deviation Apply a quadratic penalty to the vertical position error (46); Eliminate severe high-frequency jitter in the target speed and rigidly lock the UAV at the pre-calculated minimum disturbance torque altitude. On the surface.

[0090] Micro-reward mechanism of control (Critic 1): (47); in: The speed tracking weight represents the degree of penalty imposed by the system for deviations from the target speed set by the higher layer. The angular velocity suppression weight represents the system's penalty for erratic aircraft movement and high-frequency oscillations. The attitude tilt weight represents the system's penalty for large-angle tilts; The motion smoothing weight represents the system's penalty for sudden increases or decreases in engine speed commands; for The instantaneous motor motion vector for The instantaneous motor motion vector.

[0091] The sole task of controlling the network is "absolute obedience and self-stabilization." This is achieved by punishing speed deviations to ensure the network adheres strictly to commands issued from higher levels. By penalizing high-frequency angular velocities To suppress fuselage vibration caused by wind field; by constraining the real part of quaternions Strictly prevent rollovers and overturning in confined spaces; final penalty motor command jump ( This is to prevent high-frequency commutation from burning out the ESC hardware.

[0092] S4.5: Three-dimensional absolute topological mapping of inspection defects.

[0093] This module operates in parallel with the hierarchical RL control loop. When the reinforcement learning agent autonomously drives the drone to fly safely within the pipe: The system extracts the high-precision global absolute position of the EKF at a fixed spatial frequency. and attitude rotation matrix If a miniature camera can extract the relative displacement of defects such as cracks and corrosion in its own camera coordinate system... The system immediately performs a rigid body transformation projection in the coordinate system: (48); Generated absolute global coordinates Synchronously fused with ToF ranging point clouds and timestamps, it automatically outputs a 3D spatial log of pipeline topology defects with centimeter-level accuracy. The system transmits data back to the navigation layer, switches the navigation model, controls the drone to hover, and uses the camera to record and output damage images simultaneously, achieving a high degree of unity between "fully autonomous navigation in blind spots" and "accurate damage assessment without omissions."

[0094] S5: Motion calculation and underlying motor execution.

[0095] Denormalization instruction mapping: Mapping the output of a neural network... Mapped to the actual physical speed range of the motor : (49); High-frequency communication and physical execution, specifically including: Airborne flight control uses low-latency digital protocols (such as DShot600) to... The command is sent to the electronic speed controller (ESC).

[0096] The four rotors change their rotational speed, and the resulting new thrust and torque interact physically with the air inside the duct again, producing new displacement and airflow disturbances. The system clock then enters... Loop back to step S1.

[0097] In summary, this invention overcomes the limitations of traditional positioning in GPS-free environments and dark pipelines by deeply integrating cross-array Time-of-Flight (ToF) ranging, Visual Inertial Odometry (VIO), and a high-frequency IMU. It innovatively introduces the standard cylindrical geometric equation of the pipeline, uses ray casting to calculate the theoretical observation distance and construct the observation Jacobian matrix, and inversely and accurately maps the low-dimensional distance residual of ToF and the velocity residual of VIO to the 15-dimensional error state space of the UAV. This method achieves high-precision, high-frequency continuous state calculation in confined spaces with extremely low computational and hardware load. To address the intense nonlinear fluid-structure interaction effects within narrow ducts, this invention, for the first time, superimposes a Poiseuille tube flow field model with a rotor airflow disturbance (ground / ceiling effect) model based on the image method, thereby solving in real time for the "optimal anti-disturbance balance height" where airflow disturbances cancel each other out. Simultaneously, by calculating the actual total torque using an observer and algebraically subtracting the theoretical output torque of the motor, the unmodeled transient turbulent disturbance torque residual is precisely extracted. This residual accurately quantifies spatial backflow and wall effects, serving as a core state variable to provide dynamic compensation for the underlying network.

[0098] To address the shortcomings of traditional fixed-parameter cascaded PID controllers, such as sluggish response and instability under strong disturbances, this invention constructs a dual-frequency reinforcement learning mechanism that decouples macroscopic navigation (10Hz) and microscopic disturbance rejection control (100Hz). The underlying control network directly absorbs the high-precision motion error state and the real-time decoupled airflow disturbance torque residuals, bypassing the traditional step-by-step calculations through position and attitude loops. Instead, it directly maps the generated Beta distributed parameters into physical speed commands recognizable by the electronic speed controller (ESC). This end-to-end architecture achieves ultra-low latency control of motor speed and enables microsecond-level active suppression of complex aerodynamic forces within the pipe.

[0099] To overcome the "black box" nature of deep reinforcement learning networks and the potential for extremely low-probability, potentially fatal violations in complex environments, this invention innovatively embeds a non-learning analytical geometry "safety shield" based on velocity obstacle (VO) theory between two-layer Actor networks. The system calculates in real-time the set of projective cones pointing from the current position to all potential collision points on the inner wall of the pipe, triggering mandatory interception when the navigation network outputs a dangerous command. Subsequently, an online quadratic programming (QP) solver is activated, rigidly projecting the out-of-bounds command to the nearest absolutely safe velocity and sending it down to the lower layer for execution. This mechanism eliminates the risk of collisions through walls, providing a physical law-level safety net for AI-based industrial inspections.

[0100] Figure 5 A pipeline inspection UAV navigation system based on reinforcement learning is shown, comprising: The data acquisition unit 501 is configured to acquire inertial measurement data, visual inertial odometry data, cross array ranging data and electronic speed control feedback data of the UAV, and construct a continuous time state vector and an error state vector. The state calculation unit 502 is configured to: perform high-frequency motion prediction based on extended Kalman filtering, using continuous-time state vector and error state vector, and asynchronously fuse visual inertial odometry data and cross array ranging data to complete measurement update and state injection, and output high-precision UAV body angular velocity, global position, body attitude and body linear velocity. The torque calculation unit 503 is configured to: calculate the comprehensive airflow disturbance torque and solve the optimal anti-disturbance altitude based on the duct wind disturbance model and the rotor airflow disturbance model, using high-precision body angular velocity, global position and ESC feedback data; and obtain the external disturbance torque by decoupling the actual total torque currently borne by the UAV and the theoretical output torque of the motor. The strategy construction unit 504 is configured to: construct navigation reinforcement learning state and control reinforcement learning state using the optimal anti-disturbance altitude, external disturbance torque, global position, body linear velocity and cross array ranging data; output control commands based on navigation reinforcement learning state and control reinforcement learning state through continuous action mapping; and obtain safe speed commands through safety shield filtering. The flight control unit 505 is configured to: use a safe speed as the tracking target, combine the control reinforcement learning state, external disturbance torque, body attitude, body angular velocity and body linear velocity, directly output motor speed from the reinforcement learning network to control the flight of the UAV, and simultaneously complete the three-dimensional absolute topology mapping of the pipeline defect based on the global position and body attitude and enter the next control loop.

[0101] It is understood that the aforementioned units can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of the present invention. The aforementioned units are based on logical functional division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of the present invention, the system may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0102] According to another embodiment of the present invention, the system of this embodiment can be constructed by running a computer program (including program code) capable of performing the steps involved in the corresponding method of the present invention on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, loaded into the aforementioned computing device through the computer-readable recording medium, and run therein.

[0103] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0104] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic cable, digital cable) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data processing device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0105] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A pipeline inspection UAV navigation method based on reinforcement learning, characterized in that, The process includes the following: Acquire inertial measurement data, visual inertial odometry data, cross array ranging data, and electronic speed control feedback data of the UAV to construct continuous-time state vectors and error state vectors; Based on extended Kalman filtering, high-frequency motion prediction is performed using inertial measurement data by utilizing continuous-time state vectors and error state vectors. Visual inertial odometry data and cross array ranging data are asynchronously fused to complete measurement updates and state injection, outputting high-precision UAV body angular velocity, global position, body attitude and body linear velocity. Based on the duct wind disturbance model and the rotor airflow disturbance model, using high-precision body angular velocity, global position and ESC feedback data, the comprehensive airflow disturbance torque is calculated and the optimal anti-disturbance altitude is solved. The external disturbance torque is obtained by decoupling the actual total torque currently borne by the UAV from the theoretical output torque of the motor. Using the optimal anti-disturbance altitude, external disturbance torque, global position, body linear velocity and cross array ranging data, a navigation reinforcement learning state and a control reinforcement learning state are constructed. Based on the navigation reinforcement learning state and the control reinforcement learning state, control commands are output through continuous action mapping. A safe speed command is obtained by filtering through a safety shield. With safe speed as the tracking target, and combining the control reinforcement learning state, external disturbance torque, body attitude, body angular velocity and body linear velocity, the reinforcement learning network directly outputs the motor speed to control the UAV flight. Simultaneously, based on the global position and body attitude, it completes the three-dimensional absolute topology mapping of the pipeline defect and enters the next control loop.

2. The pipeline-based unmanned aerial vehicle (UAV) navigation method based on reinforcement learning as described in claim 1, characterized in that, Perform high-frequency motion prediction, including: The extrapolated value of the aircraft attitude at the next moment is calculated by integrating the quaternion differential equation using the angular velocity of the aircraft after zero bias calibration. The specific force under the machine system is projected onto the global coordinate system through the rotation matrix of the machine attitude. After canceling the gravity vector, the linear velocity of the machine is obtained by the first integration and the global position is obtained by the second integration. The state covariance matrix is ​​predicted by using the state transition Jacobian matrix and the inertial measurement process noise matrix. The body attitude, body linear velocity, global position and state covariance matrix are all used for subsequent measurement updates and state injection.

3. The pipeline-based unmanned aerial vehicle (UAV) navigation method based on reinforcement learning as described in claim 1, characterized in that, Perform measurement updates and state injection, including: The velocity residual between the predicted linear velocity and the observed linear velocity of the organism is calculated using visual inertial odometry data. Using cross array ranging data, the theoretical ranging value is solved by ray projection method combined with the geometric equation of the cylindrical surface of the pipe, and the distance residual between the theoretical ranging value and the measured ranging value is calculated. The velocity and distance residuals are mapped to the error states through the corresponding Jacobian matrices. The Kalman gain is then calculated to complete state compensation and covariance update. The resulting global position, body attitude, body angular velocity, and body linear velocity are used for subsequent torque calculation and reinforcement learning state construction.

4. The pipeline-based unmanned aerial vehicle (UAV) navigation method based on reinforcement learning as described in claim 1, characterized in that, Solving for the optimal disturbance rejection height includes: Establish a pipeline wind disturbance model corresponding to fully developed pipeline flow, and a rotor ground effect and ceiling effect disturbance model based on the image method; The disturbance moment generated by the duct wind disturbance model and the disturbance moment generated by the rotor airflow disturbance model are superimposed to obtain the comprehensive airflow disturbance moment. The optimization objective is to minimize the sum of squares of the comprehensive airflow disturbance moment. The optimal anti-disturbance height is solved by setting the derivative to zero or by using the gradient descent method to find the minimum airflow disturbance in the duct. The optimal anti-disturbance height is used for reinforcement learning navigation state construction and altitude tracking constraint calculation.

5. The pipeline-based unmanned aerial vehicle (UAV) navigation method based on reinforcement learning as described in claim 1, characterized in that, Decoupling yields the external disturbance torque, including: The angular acceleration is obtained by numerically differentiating and low-pass filtering the body angular velocity output by the extended Kalman filter, and then the total torque actually experienced by the UAV is calculated by combining the body diagonal inertia tensor. The theoretical output torque of the motor is calculated based on the motor speed, lift coefficient, torque coefficient and lever arm length in the ESC feedback data. The difference between the actual total torque currently experienced by the drone and the theoretical output torque of the motor is used to decouple and obtain the external disturbance torque of the unmodeled airflow in the duct. The external disturbance torque is used for reinforcement learning state construction and speed control calculation.

6. The pipeline-based UAV navigation method for inspection as described in claim 1, characterized in that, Construct navigation reinforcement learning states and control reinforcement learning states and output control commands, including: The navigation reinforcement learning state is constructed by fusing cross-array ranging data, body linear velocity, optimal anti-disturbance altitude, global position, and the previous cycle velocity command. The system's speed error, body attitude, body angular velocity, external disturbance torque, and motor speed at the previous moment are used to construct the control reinforcement learning state. The navigation reinforcement learning state is input into the navigation network, and a three-dimensional velocity command is output through continuous motion mapping. The control reinforcement learning state is input into the control network, and a motor speed command is output through continuous motion mapping. The three-dimensional velocity command is used for safety shield filtering, and the motor speed command is used for direct control of the UAV.

7. The pipeline-based UAV navigation method for inspection as described in claim 6, characterized in that, The security shield filtering process generates a security speed command, including: Based on the velocity obstacle theory, the inner wall of the pipe is modeled as a static obstacle and a collision cone is constructed using global position and cross array ranging data. When the three-dimensional velocity command output by the navigation network falls into the collision cone, the quadratic programming solver is activated. With the goal of minimizing the distance to the dangerous command, the safe speed is calculated within the safe speed range and used as the tracking target of the control network. The safe speed is used for the velocity error calculation of the control network and the construction of the control reinforcement learning state.

8. The pipeline-based unmanned aerial vehicle (UAV) navigation method based on reinforcement learning as described in claim 1, characterized in that, Complete the three-dimensional absolute topology mapping of pipeline defects, including: Extract the global position and body attitude from the extended Kalman filter output at a fixed spatial frequency; The relative position of the pipe defect in the camera coordinate system is projected through a rigid body transformation of the coordinate system, and the global absolute position of the defect is obtained by combining the global position and the body attitude. The global absolute position of the defect is fused with the ranging point cloud and timestamp to output a 3D topology defect log of the pipeline network and trigger UAV hovering recording. The global absolute position of the defect is used for navigation model switching and hovering control.

9. The pipeline-based unmanned aerial vehicle (UAV) navigation method based on reinforcement learning as described in claim 1, characterized in that, Controlling the drone's flight and entering the next control cycle includes: The control reinforcement learning state and safe speed are input into the reinforcement learning control network, which outputs the corresponding motor speed command. This command is then sent to the ESC via a low-latency digital protocol to drive the four rotors to adjust their speed. The interaction between rotor thrust and torque and airflow within the duct generates new displacements and disturbances. The control clock then enters the next cycle and returns to execute the multimodal sensor data acquisition step. The newly generated displacements and disturbances are used for the inertial measurement and cross array ranging data acquisition in the next cycle.

10. A pipeline inspection unmanned aerial vehicle (UAV) navigation system based on reinforcement learning, characterized in that, include: The data acquisition unit is configured to acquire inertial measurement data, visual inertial odometry data, cross array ranging data, and electronic speed control feedback data of the UAV, and construct a continuous-time state vector and an error state vector. The state calculation unit is configured to: perform high-frequency motion prediction based on extended Kalman filtering, using continuous-time state vectors and error state vectors, asynchronously fuse visual inertial odometry data and cross array ranging data to complete measurement updates and state injection, and output high-precision UAV body angular velocity, global position, body attitude and body linear velocity. The torque calculation unit is configured to: calculate the comprehensive airflow disturbance torque and solve the optimal anti-disturbance altitude based on the duct wind disturbance model and the rotor airflow disturbance model, using high-precision body angular velocity, global position and ESC feedback data; and obtain the external disturbance torque by decoupling the actual total torque currently borne by the UAV from the theoretical output torque of the motor. The strategy construction unit is configured to: construct navigation reinforcement learning state and control reinforcement learning state using the optimal anti-disturbance altitude, external disturbance torque, global position, body linear velocity and cross array ranging data; output control commands based on navigation reinforcement learning state and control reinforcement learning state through continuous action mapping; and obtain safe speed commands through safety shield filtering. The flight control unit is configured to: use a safe speed as the tracking target, combine the control reinforcement learning state, external disturbance torque, body attitude, body angular velocity and body linear velocity, and directly output motor speed to control the flight of the UAV by the reinforcement learning network. Simultaneously, it completes the three-dimensional absolute topology mapping of the pipeline defect based on the global position and body attitude and enters the next control loop.

Images