Autonomous inspection and return control method and system for blow-off pipeline robot

Abstract

The invention discloses an autonomous inspection and homeward voyage control method and system for a blow-off pipeline robot, belongs to the technical field of intelligent pipeline detection and robot control, and aims to solve the technical problems of how to realize a stable, reliable and continuous clogging exploration task in a complex blow-off pipeline environment and realize safe self-rescue in an abnormal state. According to the technical scheme, the method comprises the following steps: data dimension reduction: receiving high-dimensional environment and motion state data from a multi-modal sensor, performing real-time dimension reduction on the high-dimensional environment and motion state data by adopting a dynamic principal component analysis method, extracting key features of state information of a robot and internal state information of a pipeline, and constructing a low-dimensional state vector; performing deep reinforcement learning control based on a liquid neural network: modeling dynamic environment state change by using the liquid neural network, and realizing path planning and control in combination with a deep deterministic strategy gradient algorithm; tracing the track; and a return flight strategy based on a graph attention network and reinforcement learning.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent pipeline detection and robot control, and in particular to an autonomous inspection and return control method and system for a sewage pipeline robot. Background Art

[0002] Currently, sewage pipeline inspections and blockage detection rely primarily on manual access or remotely controlled robotic systems. These systems suffer from low efficiency and high operational risks, and their control accuracy and ability to complete tasks are severely limited when navigating long, winding, and complex underground pipeline networks. Existing technologies are particularly unable to achieve reliable and continuous operation in extreme environments, such as those with high pollution, low light levels, and frequent water level fluctuations.

[0003] With the development of deep reinforcement learning and mobile robotics, autonomous decision-making and intelligent control have become research hotspots. While existing methods have attempted to incorporate deep reinforcement learning into pipeline inspection tasks, traditional neural network models have poor adaptability to dynamic environments and are prone to decision-making delays and even control failures when faced with high-dimensional state inputs and uncertain environmental dynamics. Furthermore, sewage pipeline robots are typically deployed on resource-constrained embedded platforms with limited computing resources and power supplies, making it difficult to support the high-frequency operations of large-scale neural networks. Furthermore, robots often face challenges such as complex return paths, loss of historical trajectories, and sensor drift during mission execution, making safe return difficult in situations such as low battery levels and positioning interruptions.

[0004] Therefore, how to achieve stable, reliable and continuous blockage detection tasks in a complex sewage pipeline environment and have safe self-rescue capabilities under abnormal conditions is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The technical task of the present invention is to provide a sewage pipe robot autonomous inspection and return control method and system to solve the problem of how to achieve stable, reliable and continuous blockage detection tasks in a complex sewage pipe environment, and have safe self-rescue in abnormal conditions.

[0006] The technical task of the present invention is achieved in the following manner: a method for autonomous inspection and return control of a sewage pipe robot, the method being as follows:

[0007] Data dimensionality reduction: Receive high-dimensional environmental and motion state data from multimodal sensors and use dynamic principal component analysis (Dynamic PCA) to perform real-time dimensionality reduction on the high-dimensional environmental and motion state data. This extracts key features of the robot's own state information and the pipeline's internal state information, and constructs a low-dimensional state vector.

[0008] Deep reinforcement learning control based on liquid neural networks: This approach uses liquid neural networks (LNNs) to model dynamic environmental state changes, combined with the Deep Deterministic Policy Gradient (DDPG) algorithm for path planning and control. It also defines the state space, or action space, and designs a multi-factor reward function to guide the robot in completing tasks.

[0009] Trajectory tracing: The robot's historical motion trajectory in the pipeline is recorded, and trajectory compression is performed using a spatial change threshold and trajectory interpolation algorithm to retain key path feature points. Key event nodes, such as pollution sources and frequent obstacles, are also identified and stored, ensuring high-precision path reconstruction and event backtracking even under limited memory resources.

[0010] Return strategy based on graph attention network and reinforcement learning (GAT+DRL): When the robot needs to return, a trajectory graph model is constructed based on the robot's historical motion trajectory. Key path nodes and trajectory compression points are defined as nodes in the graph, and trajectory sequence and feasible paths are used as edges of the graph. The graph attention network (GAT) is combined to update the node embedding, enhance the perception ability of key areas, and plan the return path based on the reinforcement learning algorithm to improve the efficiency and safety of the return.

[0011] As a preferred method, data dimensionality reduction is as follows:

[0012] Collect the robot's own state information and the internal state information of the pipeline; the robot's own state information includes the current linear velocity, current angular velocity, posture information, three-axis acceleration, three-axis angular acceleration, task completion status, and the history of the previous steps; the internal state information of the pipeline includes the distance to obstacles in each direction, local slope, pipeline curvature, and pass probability;

[0013] Construct the original state vector s t , specifically: s t =[robot status, pipeline internal environment status];

[0014] The original state vector s is transformed into t Perform dimensionality reduction to obtain the low-dimensional state vector x t , the formula is: in, represents the mean of the original state vector; Represents the matrix used to project the original state vector, and the original state vector is projected into a k-dimensional space through the covariance matrix.

[0015] As a preferred method, the deep reinforcement learning control based on liquid neural network is as follows:

[0016] Define the state space: The state space is defined as a low-dimensional state vector x t ;

[0017] Define the action space: The linear velocity and angular velocity range of the current action output is defined as the action space, expressed as a t =[v t ,ω t ]; where v represents linear velocity, v∈[-2,2], in meters per second; ω represents angular velocity, ω∈[-1,1], in rad per second;

[0018] Design a reward function: This integrates multiple factors to guide the robot to move forward efficiently, avoid obstacles, and complete exploration tasks. The formula is as follows:

[0019] R t =W1×r forward +W2×r safe +W3×r task +W4×r smooth ;

[0020] Among them, r forward Represents the reward value for encouraging movement in the target direction; r safe Indicates the reward value for maintaining a safe distance and avoiding obstacles; r task represents the task completion degree of the pluggable perception module, that is, the target task completion rate; r smooth Represents the oscillation penalty. The larger the reward value, the more drastic the change, that is, the more severe the "jitter", which requires a penalty. W1, W2, W3, and W4 are the weights of the reward value for encouraging movement in the target direction, the weights of the reward value for maintaining a safe distance and avoiding obstacles, the weight of the task completion of the pluggable perception module, and the weight of the oscillation penalty, respectively. Before calculating the reward value for encouraging movement in the target direction, the reward value for maintaining a safe distance and avoiding obstacles, the task completion of the pluggable perception module, and the oscillation penalty, normalization processing must be performed, as follows:

[0021] r forward =V t ×cos(θ t );

[0022] Among them, V t Indicates the current speed of the robot; θ t Indicates the angle between the robot's current moving direction and the target direction; cos(θ t ) indicates whether the movement is towards the target. The closer to 1, the better. If the movement is towards the target, the reward is positive; if the movement is in the opposite direction, the reward is negative.

[0023]

[0024] Among them, d t Indicates the distance between the robot and the nearest obstacle; d safeIndicates the set minimum safety distance threshold. If you are too close to an obstacle, you will receive a negative reward.

[0025]

[0026] Among them, n t Indicates the number of tasks actually completed in the time step; n total Indicates the number of tasks expected to be completed this time;

[0027] r smooth =(a t -a t-1 ) 2 ;

[0028] Among them, a t Indicates the action at the current time step; a t-1 Represents the action at the previous time step.

[0029] Preferably, the liquid neural network includes a state encoder, an actor network, a critic network, an actor target network, and a critic target network;

[0030] The state encoder (LNN, Liquid Neural Network) is used to process complex time series inputs and model the impact of state history on current decisions. The dynamic update formula of the state encoder is as follows:

[0031]

[0032] z t =u t ;

[0033] Among them, x t represents the low-dimensional state vector; μ t represents the internal state of the neuron; Δt represents the discrete time step; τ(x t ) represents the state-related time constant, allowing the network to respond to different inputs at different speeds; W in Represents the input weight matrix; W rec Represents the weight matrix between recursive connections (hidden layers); b represents the bias vector; f(.) represents the nonlinear activation function, and the final output z t is the encoded state, used for subsequent strategy evaluation;

[0034] The Actor network is used to output actions based on the state encoder. The formula is as follows:

[0035] a t =μ(z t |θ μ );

[0036] Among them, μ represents the policy function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action, that is, the value of linear velocity and angular velocity; θ μ Represents the learnable parameters of the Actor network;

[0037] The Critic network is used as an action value function to evaluate the quality of the current state-action pair, expressed as: Q(z t ,a t |θ Q );

[0038] Where Q(·) represents the action value function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q Represents the parameters of the Critic network, which is used to update the Actor and serve as the basis for estimating the objective function return;

[0039] The Actor target network and the Critic target network are used to construct stable target values. Specifically, the slowly updated Actor target network and Critic target network are introduced through the deep deterministic policy gradient algorithm, corresponding to the copies of the Actor and Critic respectively, and are defined as: μ′≈μ,Q′≈Q;

[0040] Among them, μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; Q′ represents the Critic target network.

[0041] Preferably, the training process of the liquid neural network is as follows:

[0042] The agent interacts with the environment: During each interaction, the agent selects an action based on its current state, and the environment feeds back the new state and reward. The "state-action-reward-new state" sequence is recorded as data for subsequent learning.

[0043] The TD target is used to update the Critic network and represents "current reward + estimate of future reward". The TD target calculation formula is as follows:

[0044] y t =r t +γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ );

[0045] Among them, y t Represents the TD target value of this step, which is used to train the Critic network; rt represents the immediate reward obtained at the current moment t; γ represents the discount factor, which ranges from 0 to 1 and measures the importance of future rewards; Q′ represents the Critic target network, which is used to estimate the action value of the next state; z t+1 Represents the state characteristics of the next moment; μ′ represents the Actor target network, which is used to give the optimal action in the next state; θ μ′ Represents the parameters of the Actor target network; θ Q′ Represents the parameters of the Critic target network;

[0046] Minimize the loss: L Q =(Q(z t ,a t )-y t ) 2 ;

[0047] Among them, L Q Represents the loss function of the Critic network, which measures the gap between the current Q value and the TD target; Q(z t ,a t ) represents the current Critic network state z t and action a t Estimated value of

[0048] The Actor network updates its parameters by maximizing the Q value of the Critic network's output action. The formula is as follows:

[0049]

[0050] in, Represents the objective function gradient of the Actor network parameters, which is used to update the Actor parameters; Q(z,a|θ Q ) represents the value estimate of the current Critic network for any state z and action a, and the parameter is θ Q ; represents the gradient of Q value with respect to action a, which measures the impact of action change on value; a = μ(z) means replacing a with the action output by the Actor network; Represents the gradient of the Actor network output to its own parameters; θ μ Represents the parameters of the Actor network;

[0051] Soft update of the target network (Polyak averaging) allows the target network parameters to slowly follow the changes in the main network parameters to improve training stability. The formula is as follows:

[0052] θ Q′ ←τθ Q +(1-τθ Q′ ;

[0053] θ μ′ ←τθ μ +(1-τ)θ μ′ ;

[0054] Among them, τ∈(0,1) represents the soft update rate, which takes a value of 0.005 and is used to smooth parameter updates to prevent oscillations.

[0055] As a preferred option, the traceability is as follows:

[0056] Real-time recording and compressed storage of trajectories: Assume that the position of the robot at time t is X t , and its trajectory sequence is expressed as: τ={X1,X2,…,X n};

[0057] In order to reduce the memory burden, linear interpolation combined with key point preservation algorithm is used to compress the trajectory. The formula is: Among them, δ represents the spatial variation threshold, and the compression function keeps the path accuracy error no more than the preset value;

[0058] Key node marking and storage: The robot uses sensors to identify the pollution source point P 污染源 ={p 污染源i}、Obstacle source hotspot P 障碍源 ={p 障碍源i}, the key node set is defined as: k = P 污染源 ∪P 障碍源 The key node set will be stored first and will not be affected by trajectory compression, so as to provide reference for return planning and replay after the inspection mission is completed;

[0059] Trajectory reconstruction capability: Using compressed trajectories and key node sets to perform approximate trajectory restoration, we can achieve a near-true path tracing capability, which is expressed as:

[0060] As a preferred option, the return strategy based on graph attention network and reinforcement learning is as follows:

[0061] Environment graph construction: Construct pipeline graph model G = (V, E); where the node set V contains trajectory compression points All positions and key nodes k in The construction method ensures that key nodes are perceived first, improving the interpretability and safety of the return path; the edge set E consists of trajectory sequences or feasible paths, and is assigned obstacle risk and distance weights;

[0062] Graph Attention Embedding Extraction: LeakyReLU (Leaky Rectified Linear Unit) is a commonly used neural network activation function, expressed as: Among them, α is a constant with a value of 0.01; the graph attention mechanism is used to update the node embedding, and each node embedding h i The transformation formula is as follows:

[0063]

[0064] Among them, α ij represents the attention weight of node j to node i; W represents the linear transformation matrix of the graph attention layer, which is used to project the original features into the new feature space; h i represents the original feature vector of node i; a represents the parameter vector of the attention mechanism, which is used to score the correlation between node pairs; N(i) represents the set of neighbor nodes of node i; the node feature update formula is: i i=σ(∑ j∈N(i) α ij Wh j ); where h i ′ represents the updated node features, integrating neighbor node information; σ represents the activation function; W represents the learnable weight matrix for feature transformation; h j Represents the current feature vector of neighbor node j;

[0065] Reinforcement learning modeling, as follows:

[0066] State space design: state vector s t By the current position node h i i and the environmental state, the formula is as follows:

[0067]

[0068] Among them, h i ′ represents the embedded representation of the current node after GAT encoding; θ t Indicates the current heading angle; e t Indicates the current remaining energy (scalar); Indicates the distance from the current position to the return target point; rask t Indicates the risk value of the current location or path;

[0069] Action space design: The linear velocity and angular velocity range of the current action output is defined as the action space, expressed as a t =[v t ,ω t ]; where v represents linear velocity, v∈[-2,2], in meters per second; ω represents angular velocity, ω∈[-1,1], in rad per second;

[0070] The reward function is designed with the following formula:

[0071]

[0072] Among them: i represents the reward weight; Indicates the distance from the current position to the return target point; rask t Indicates the risk value of the current location or path; collision t Indicates whether a collision has occurred; isdone indicates whether the return target point has been reached;

[0073] Policy network design: including state encoder, actor network, critic network, actor target network and critic target network.

[0074] Preferably, the pluggable sensing module has the following functions:

[0075] ① Supports various heterogeneous sensor modules and can provide feedback on task rewards to the main control module, improving collaborative perception and decision-making capabilities; heterogeneous sensor modules include image acquisition sensors, scanning area sensors, obstacle detection sensors, and pollution identification sensors;

[0076] ② Support heterogeneous task modules, flexible mounting and unmounting, and decoupling from the main control system (vehicle control); among them, heterogeneous task modules include pollution detection module, crack identification module and temperature monitoring module;

[0077] ③ Unified interface specifications, support module registration / deregistration;

[0078] ④ Output task results or events (such as "pollution detected");

[0079] ⑤ Feedback reward signal (passed to the DRL module).

[0080] A sewage pipe robot autonomous inspection and return control system, the system is used to implement the sewage pipe robot autonomous inspection and return control method as described above; the system includes:

[0081] The data dimensionality reduction module receives high-dimensional environmental and motion state data from multimodal sensors and uses dynamic principal component analysis (Dynamic PCA) to perform real-time dimensionality reduction on the high-dimensional environmental and motion state data, extracting key features of the robot's own state information and the pipeline's internal state information to construct a low-dimensional state vector.

[0082] A deep reinforcement learning control module based on liquid neural networks is used to model dynamic environmental state changes using liquid neural networks (LNNs). It combines the deep deterministic policy gradient (DDPG) algorithm to implement path planning and control, define the state space (i.e., action space), and design a multi-factor reward function to guide the robot to complete the task.

[0083] The trajectory tracing module records the robot's historical motion trajectory in the pipeline. It combines spatial change thresholds with a trajectory interpolation algorithm to compress the trajectory, retain key path feature points, and identify and store key event nodes such as pollution sources and frequent obstacles. This ensures high-precision path reconstruction and event backtracking even under limited memory resources.

[0084] The return strategy module based on graph attention network and reinforcement learning (GAT+DRL) is used to build a trajectory graph model based on the robot's historical motion trajectory when the robot needs to return. The key path nodes and trajectory compression points are defined as nodes in the graph, and the trajectory sequence and feasible paths are used as edges of the graph. The graph attention network (GAT) is combined to update the node embedding, enhance the perception ability of key areas, and plan the return path based on the reinforcement learning algorithm to improve the efficiency and safety of the return.

[0085] Preferably, the liquid neural network includes a state encoder, an actor network, a critic network, an actor target network, and a critic target network;

[0086] The state encoder (LNN, Liquid Neural Network) is used to process complex time series inputs and model the impact of state history on current decisions. The dynamic update formula of the state encoder is as follows:

[0087]

[0088] z t =u t ;

[0089] Among them, x t represents the low-dimensional state vector; μ t represents the internal state of the neuron; Δt represents the discrete time step; τ(x t ) represents the state-related time constant, allowing the network to respond to different inputs at different speeds; W in Represents the input weight matrix; W rec Represents the weight matrix between recursive connections (hidden layers); b represents the bias vector; f(.) represents the nonlinear activation function, and the final output z t is the encoded state, used for subsequent strategy evaluation;

[0090] The Actor network is used to output actions based on the state encoder. The formula is as follows:

[0091] a t =μ(z t |θ μ );

[0092] Among them, μ represents the policy function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action, that is, the value of linear velocity and angular velocity; θ μ Represents the learnable parameters of the Actor network;

[0093] The Critic network is used as an action value function to evaluate the quality of the current state-action pair, expressed as: Q(z t ,a t |θ Q );

[0094] Where Q(·) represents the action value function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q Represents the parameters of the Critic network, which is used to update the Actor and serve as the basis for estimating the objective function return;

[0095] The Actor target network and the Critic target network are used to construct stable target values. Specifically, the slowly updated Actor target network and Critic target network are introduced through the deep deterministic policy gradient algorithm, corresponding to the copies of the Actor and Critic respectively, and are defined as: μ′≈μ,Q ′ ≈Q;

[0096] Among them, μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; Q′ represents the Critic target network;

[0097] The training process of the liquid neural network is as follows:

[0098] The agent interacts with the environment: During each interaction, the agent selects an action based on its current state, and the environment feeds back the new state and reward. The "state-action-reward-new state" sequence is recorded as data for subsequent learning.

[0099] The TD target is used to update the Critic network and represents "current reward + estimate of future reward". The TD target calculation formula is as follows:

[0100] y t =r t+γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ );

[0101] Among them, y t Represents the TD target value of this step, which is used to train the Critic network; r t represents the immediate reward obtained at the current moment t; γ represents the discount factor, which ranges from 0 to 1 and measures the importance of future rewards; Q′ represents the Critic target network, which is used to estimate the action value of the next state; z t+1 Represents the state characteristics of the next moment; μ′ represents the Actor target network, which is used to give the optimal action in the next state; θ μ′ Represents the parameters of the Actor target network; θ Q′ Represents the parameters of the Critic target network;

[0102] Minimize the loss: L Q =(Q(z t ,a t )-y t ) 2 ;

[0103] Among them, L Q Represents the loss function of the Critic network, which measures the gap between the current Q value and the TD target; Q(z t ,a t ) represents the current Critic network state z t and action a t Estimated value of

[0104] The Actor network updates its parameters by maximizing the Q value of the Critic network's output action. The formula is as follows:

[0105]

[0106] in, Represents the objective function gradient of the Actor network parameters, which is used to update the Actor parameters; Q(z,a|θ Q ) represents the value estimate of the current Critic network for any state z and action a, and the parameter is θ Q ; represents the gradient of Q value with respect to action a, which measures the impact of action change on value; a = μ(z) means replacing a with the action output by the Actor network; Represents the gradient of the Actor network output to its own parameters; θ μ Represents the parameters of the Actor network;

[0107] Soft update of the target network (Polyak averaging) allows the target network parameters to slowly follow the changes in the main network parameters to improve training stability. The formula is as follows:

[0108] θ Q′ ←τθ Q +(1-τθ Q′ ;

[0109] θ μ′ ←τθ μ +(1-τ)θ μ′ ;

[0110] Among them, τ∈(0,1) represents the soft update rate, which takes a value of 0.005 and is used to smooth parameter updates to prevent oscillations.

[0111] The sewage pipe robot autonomous inspection and return control method and system of the present invention have the following advantages:

[0112] (1) The present invention uses multimodal sensors to collect pipeline environmental data and implements real-time dimensionality reduction processing of high-dimensional perception data through a dynamic principal component analysis module, thereby extracting key environmental features and improving the system's decision-making efficiency and response speed. The liquid neural network has excellent temporal modeling capabilities and can effectively adapt to the dynamic changes in the environmental state in the sewage pipeline, enhancing the robot's path planning and obstacle avoidance capabilities in complex environments. In addition, the intelligent control strategy designed in combination with the deep deterministic policy gradient (DDPG) algorithm achieves highly robust and adaptive path decision-making.

[0113] (2) This invention introduces a trajectory tracing module and an autonomous return-to-home mechanism based on DDPG, which solves the problem of the robot being unable to return safely under extreme working conditions such as insufficient power or communication interruption. This significantly improves the practicality and safety of the system. It is suitable for scenarios such as underground sewage pipe networks and industrial wastewater channels, and has broad application prospects and important engineering value in improving inspection efficiency and intelligence level.

[0114] (3) The intelligent robot system of the present invention integrates efficient perception, intelligent decision-making, autonomous return and trajectory tracing capabilities. It can perform stable, reliable and continuous blockage detection tasks in complex sewage pipeline environments and has the ability to safely self-rescue in abnormal conditions. To address the problems of complex underground sewage pipeline environments, difficult access and limited communication, the system integrates multiple intelligent algorithms to achieve integrated perception and decision-making and full-process autonomous control of the robot in restricted environments.

[0115] (IV) The present invention proposes an autonomous sewage pipe exploration robot system that integrates dynamic principal component analysis (Dynamic PCA), liquid neural network (LNN) and deep reinforcement learning (DRL). It combines a differential motion model to achieve intelligent path control. To solve the return problem during the exploration process, the present invention designs a trajectory tracing module and integrates a graph attention network (GAT) and reinforcement learning method to formulate an efficient return strategy. The system structure of the present invention is modular and highly intelligent, which can significantly reduce the complexity of manual operation and improve the success rate and autonomy of sewage pipe blockage exploration tasks. It is suitable for complex and dynamic underground pipe network environments.

[0116] (5) The present invention uses a liquid neural network to model time series state data. Its neuron state is dynamically updated through continuous time differentiation. It has a memory capability based on state history and can efficiently learn control strategies in dynamic environments, thereby improving the robot's stability and generalization ability in complex pipeline environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0117] The present invention will be further described below with reference to the accompanying drawings.

[0118] Attachment Figure 1 This is a schematic diagram of the structure of the sewage pipe robot's autonomous inspection and return control system;

[0119] Attachment Figure 2 This is a flow chart of the cruise state;

[0120] Attachment Figure 3 This is a flowchart of trajectory tracing;

[0121] Attachment Figure 4 This is a flowchart of the return state;

[0122] Attachment Figure 5 Schematic diagram of the structure of the Liquid Neural Network (LNN) and Deep Reinforcement Learning (DRL) model. DETAILED DESCRIPTION

[0123] The autonomous inspection and return control method and system of the sewage pipe robot of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments.

[0124] Example 1:

[0125] This embodiment provides a sewage pipe robot autonomous inspection and return control method, the method is as follows:

[0126] S1. Data Dimensionality Reduction: Receive high-dimensional environment and motion state data from multimodal sensors, perform real-time dimensionality reduction on the data using Dynamic Principal Component Analysis (Dynamic PCA), extract key features of the robot's own state information and the pipeline's internal state information, and construct a low-dimensional state vector.

[0127] S2. Deep reinforcement learning control based on liquid neural networks: This approach uses liquid neural networks (LNNs) to model dynamic environmental state changes, combined with the deep deterministic policy gradient (DDPG) algorithm to implement path planning and control. It also defines the state space, or action space, and designs a multi-factor reward function to guide the robot in completing tasks.

[0128] S3, Trajectory Tracing: This records the robot's historical motion trajectory in the pipeline, compresses the trajectory using a spatial change threshold and trajectory interpolation algorithm, retains key path feature points, and identifies and stores key event nodes such as pollution sources and frequent obstacles. This ensures high-precision path reconstruction and event tracing even under limited memory resources.

[0129] S4. Return strategy based on graph attention network and reinforcement learning (GAT+DRL): When the robot needs to return, a trajectory graph model is constructed based on the robot's motion history. Key path nodes and trajectory compression points are defined as nodes in the graph, and trajectory sequence and feasible paths are used as edges of the graph. The graph attention network (GAT) is combined to update the node embedding, enhance the perception ability of key areas, and plan the return path based on the reinforcement learning algorithm to improve the efficiency and safety of the return.

[0130] The data dimensionality reduction in step S1 of this embodiment is specifically as follows:

[0131] S101. Collect the robot's own state information and the internal state information of the pipeline. The robot's own state information includes the current linear velocity, current angular velocity, posture information, three-axis acceleration, three-axis angular acceleration, task completion status, and the previous several steps of historical action. The internal state information of the pipeline includes the obstacle distance in each direction, local slope, pipeline curvature, and pass probability. The robot's own motion state is shown in Table 1, and the internal state of the pipeline is shown in Table 2.

[0132] Table 1 Self-motion state table

[0133]

[0134] Table 2 Pipeline internal status table

[0135]

[0136] S102, construct the original state vector s t , specifically: s t =[robot status, pipeline internal environment status];

[0137] S103, the original state vector s is converted into t Perform dimensionality reduction to obtain the low-dimensional state vector x t , the formula is: in, represents the mean of the original state vector; Represents the matrix used to project the original state vector, and the original state vector is projected into a k-dimensional space through the covariance matrix.

[0138] The deep reinforcement learning control based on the liquid neural network in step S2 of this embodiment is specifically as follows:

[0139] S201. Define state space: define the state space as a low-dimensional state vector x t ;

[0140] S202, define the action space: define the linear velocity and angular velocity range of the current action output as the action space, expressed as a t =[v t ,ω t ]; where v represents linear velocity, v∈[-2,2], in meters per second; ω represents angular velocity, ω∈[-1,1], in rad per second;

[0141] S203. Design a reward function: This function integrates multiple factors to guide the robot to move forward efficiently, avoid obstacles, and complete exploration tasks. The formula is as follows:

[0142] R t =W1×r forward +W2×r safe +W3×r task +W4×r smooth ;

[0143] Among them, r forward Represents the reward value for encouraging movement in the target direction; r safe Indicates the reward value for maintaining a safe distance and avoiding obstacles; r task represents the task completion degree of the pluggable perception module, that is, the target task completion rate; r smoothRepresents the oscillation penalty. The larger the reward value, the more drastic the change, that is, the more severe the "jitter", which requires a penalty. W1, W2, W3, and W4 are the weights of the reward value for encouraging movement in the target direction, the weights of the reward value for maintaining a safe distance and avoiding obstacles, the weight of the task completion of the pluggable perception module, and the weight of the oscillation penalty, respectively. Before calculating the reward value for encouraging movement in the target direction, the reward value for maintaining a safe distance and avoiding obstacles, the task completion of the pluggable perception module, and the oscillation penalty, normalization processing must be performed, as follows:

[0144] r forward =V t ×cos(θ t );

[0145] Among them, V t Indicates the current speed of the robot; θ t Indicates the angle between the robot's current moving direction and the target direction; cos(θ t ) indicates whether the movement is towards the target. The closer to 1, the better. If the movement is towards the target, the reward is positive; if the movement is in the opposite direction, the reward is negative.

[0146]

[0147] Among them, d t Indicates the distance between the robot and the nearest obstacle; d safe Indicates the set minimum safety distance threshold. If you are too close to an obstacle, you will receive a negative reward.

[0148]

[0149] Among them, n t Indicates the number of tasks actually completed in the time step; n total Indicates the number of tasks expected to be completed this time;

[0150] r smooth =(a t -a t-1 ) 2 ;

[0151] Among them, a t Indicates the action at the current time step; a t-1 Represents the action at the previous time step.

[0152] Compared with traditional RNN and LSTM, the liquid neural network in step S2 of this embodiment has the following key advantages: time continuity modeling: describing neuron dynamics through differential equations can more naturally simulate state evolution under continuous time; memory ability of state changes: neuron states have dynamic memory, which is suitable for processing the dependence of state sequences in navigation tasks; high data efficiency: effective strategies can still be learned in a small sample environment; strong stability: avoiding problems such as traditional RNN learning instability and gradient explosion / vanishing.

[0153] As attached Figure 2 and 5 As shown, the liquid neural network in this embodiment includes a state encoder, an actor network, a critic network, an actor target network, and a critic target network;

[0154] The state encoder (LNN, Liquid Neural Network) is used to process complex time series inputs and model the impact of state history on current decisions. The dynamic update formula of the state encoder is as follows:

[0155]

[0156] z t =u t ;

[0157] Among them, x t represents the low-dimensional state vector; μ t represents the internal state of the neuron; Δt represents the discrete time step; τ(x t ) represents the state-related time constant, allowing the network to respond to different inputs at different speeds; W in Represents the input weight matrix; W rec Represents the weight matrix between recursive connections (hidden layers); b represents the bias vector; f(.) represents the nonlinear activation function, and the final output z t is the encoded state, used for subsequent strategy evaluation;

[0158] The Actor network is used to output actions based on the state encoder. The formula is as follows:

[0159] a t =μ(z t |θ μ );

[0160] Among them, μ represents the policy function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action, that is, the value of linear velocity and angular velocity; θ μ Represents the learnable parameters of the Actor network;

[0161] The Critic network is used as an action value function to evaluate the quality of the current state-action pair, expressed as: Q(z t ,a t |θ Q );

[0162] Where Q(·) represents the action value function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q Represents the parameters of the Critic network, which is used to update the Actor and serve as the basis for estimating the objective function return;

[0163] The Actor target network and the Critic target network are used to construct stable target values. Specifically, the slowly updated Actor target network and Critic target network are introduced through the deep deterministic policy gradient algorithm, corresponding to the copies of the Actor and Critic respectively, and are defined as: μ′≈μ,Q′≈Q;

[0164] Among them, μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; Q′ represents the Critic target network.

[0165] The training process of the liquid neural network in step S2 of this embodiment is specifically as follows:

[0166] (1) The agent interacts with the environment: During each interaction, the agent selects an action based on the current state, and the environment feeds back the new state and reward. The “state-action-reward-new state” is recorded as data for subsequent learning.

[0167] (2) The TD target is used to update the Critic network, which represents "current reward + estimate of future reward". The TD target calculation formula is as follows:

[0168] y t =r t +γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ );

[0169] Among them, y t Represents the TD target value of this step, which is used to train the Critic network; r t represents the immediate reward obtained at the current moment t; γ represents the discount factor, which ranges from 0 to 1 and measures the importance of future rewards; Q′ represents the Critic target network, which is used to estimate the action value of the next state; zt+1 Represents the state characteristics of the next moment; μ′ represents the Actor target network, which is used to give the optimal action in the next state; θ μ′ Represents the parameters of the Actor target network; θ Q′ Represents the parameters of the Critic target network;

[0170] (3) Minimize the loss: L Q =(Q(z t ,a t )-y t ) 2 ;

[0171] Among them, L Q Represents the loss function of the Critic network, which measures the gap between the current Q value and the TD target; Q(z t ,a t ) represents the current Critic network state z t and action a t Estimated value of

[0172] (4) The Actor network updates its parameters by maximizing the Q value of the Critic network's output action. The formula is as follows:

[0173]

[0174] in, Represents the objective function gradient of the Actor network parameters, which is used to update the Actor parameters; Q(z,a|θ Q ) represents the value estimate of the current Critic network for any state z and action a, and the parameter is θ Q ; represents the gradient of Q value with respect to action a, which measures the impact of action change on value; a = μ(z) means replacing a with the action output by the Actor network; Represents the gradient of the Actor network output to its own parameters; θ μ Represents the parameters of the Actor network;

[0175] (5) Soft update of the target network (Polyak averaging). The target network parameters slowly follow the changes of the main network parameters to improve the training stability. The formula is as follows:

[0176] θ Q′ ←τθ Q +(1-τθ Q′ ;

[0177] θ μ′ ←τθ μ +(1-τ)θ μ′ ;

[0178] Among them, τ∈(0,1) represents the soft update rate, which takes a value of 0.005 and is used to smooth parameter updates to prevent oscillations.

[0179] As attached Figure 3 As shown, the trajectory tracing in step S3 of this embodiment is specifically as follows:

[0180] S301, real-time recording and compressed storage of trajectory: Assume that the position of the robot at time t is X t , and its trajectory sequence is expressed as: τ={X1,X2,…,X n};

[0181] S302: To reduce the memory burden, linear interpolation combined with key point preservation algorithm is used to compress the trajectory. The formula is: Among them, δ represents the spatial variation threshold, and the compression function keeps the path accuracy error no more than the preset value;

[0182] S303, key node marking and storage: The robot identifies the pollution source point P through sensors 污染源 ={p 污染源i}、Obstacle source hotspot P 障碍源 ={p 障碍源i}, the key node set is defined as: k = P 污染源 ∪P 障碍源 The key node set will be stored first and will not be affected by trajectory compression, so as to provide reference for return planning and replay after the inspection mission is completed;

[0183] S304, trajectory reconstruction capability: Use the compressed trajectory and key node set to perform approximate trajectory restoration, achieving a nearly true path tracing capability, expressed as:

[0184] As attached Figure 4 As shown, the return strategy based on graph attention network and reinforcement learning in step S4 of this embodiment is as follows:

[0185] S401, environment graph construction: Construct pipeline graph model G = (V, E); wherein the node set V contains trajectory compression points All positions and key nodes k in The construction method ensures that key nodes are perceived first, improving the interpretability and safety of the return path; the edge set E consists of trajectory sequences or feasible paths, and is assigned obstacle risk and distance weights;

[0186] S402. Graph Attention Embedding Extraction: LeakyReLU (Leaky Rectified Linear Unit) is a commonly used neural network activation function, expressed as: Among them, α is a constant with a value of 0.01; the graph attention mechanism is used to update the node embedding, and each node embedding h i The transformation formula is as follows:

[0187]

[0188] Among them, α ij represents the attention weight of node j to node i; W represents the linear transformation matrix of the graph attention layer, which is used to project the original features into the new feature space; h i represents the original feature vector of node i; a represents the parameter vector of the attention mechanism, which is used to score the correlation between node pairs; N(i) represents the set of neighbor nodes of node i; the node feature update formula is: i ′=σ(∑ j∈N(i) α ij Wh j ); where h i ′ represents the updated node features, integrating neighbor node information; σ represents the activation function; W represents the learnable weight matrix for feature transformation; h j Represents the current feature vector of neighbor node j;

[0189] S403, reinforcement learning modeling, specifically as follows:

[0190] S40301, State Space Design: State Vector s t By the current position node h i ′ and the environmental state, the formula is as follows:

[0191]

[0192] Among them, h i ′ represents the embedded representation of the current node after GAT encoding; θ t Indicates the current heading angle; e t Indicates the current remaining energy (scalar); Indicates the distance from the current position to the return target point; rask t Indicates the risk value of the current location or path;

[0193] S40302, Action Space Design: The linear velocity and angular velocity range of the current action output is defined as the action space, expressed as a t =[v t ,ω t ]; where v represents linear velocity, v∈[-2,2], in meters per second; ω represents angular velocity, ω∈[-1,1], in rad per second;

[0194] S40303. Reward function design. The formula is as follows:

[0195]

[0196] Among them: i represents the reward weight; Indicates the distance from the current position to the return target point; rask t Indicates the risk value of the current location or path; collision t Indicates whether a collision has occurred; isdone indicates whether the return target point has been reached;

[0197] S40304, Policy Network Design: including state encoder, Actor network, Critic network, Actor target network and Critic target network;

[0198] The state encoder (LNN, Liquid Neural Network) is used to process complex time series inputs and model the impact of state history on current decisions. The dynamic update formula of the state encoder is as follows:

[0199]

[0200] z t =u t ;

[0201] Among them, x t represents the low-dimensional state vector; μ t represents the internal state of the neuron; Δt represents the discrete time step; τ(x t ) represents the state-related time constant, allowing the network to respond to different inputs at different speeds; W in Represents the input weight matrix; W rec Represents the weight matrix between recursive connections (hidden layers); b represents the bias vector; f(.) represents the nonlinear activation function, and the final output z t is the encoded state, used for subsequent strategy evaluation;

[0202] The Actor network is used to output actions based on the state encoder. The formula is as follows:

[0203] a t =μ(z t |θ μ );

[0204] Among them, μ represents the policy function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action, that is, the value of linear velocity and angular velocity; θμ Represents the learnable parameters of the Actor network;

[0205] The Critic network is used as an action value function to evaluate the quality of the current state-action pair, expressed as: Q(z t ,a t |θ Q );

[0206] Where Q(·) represents the action value function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q Represents the parameters of the Critic network, which is used to update the Actor and serve as the basis for estimating the objective function return;

[0207] The Actor target network and the Critic target network are used to construct stable target values. Specifically, the slowly updated Actor target network and Critic target network are introduced through the deep deterministic policy gradient algorithm, corresponding to the copies of the Actor and Critic respectively, and are defined as: μ′≈μ,Q′≈Q;

[0208] Among them, μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; Q′ represents the Critic target network.

[0209] The training process of the liquid neural network in this embodiment is as follows:

[0210] (1) The agent interacts with the environment: During each interaction, the agent selects an action based on the current state, and the environment feeds back the new state and reward. The “state-action-reward-new state” is recorded as data for subsequent learning.

[0211] (2) The TD target is used to update the Critic network, which represents "current reward + estimate of future reward". The TD target calculation formula is as follows:

[0212] y t =r t +γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ );

[0213] Among them, y t Represents the TD target value of this step, which is used to train the Critic network; r trepresents the immediate reward obtained at the current moment t; γ represents the discount factor, which ranges from 0 to 1 and measures the importance of future rewards; Q′ represents the Critic target network, which is used to estimate the action value of the next state; z t+1 Represents the state characteristics of the next moment; μ′ represents the Actor target network, which is used to give the optimal action in the next state; θ μ′ Represents the parameters of the Actor target network; θ Q′ Represents the parameters of the Critic target network;

[0214] (3) Minimize the loss: L Q =(Q(z t ,a t )-y t ) 2 ;

[0215] Among them, L Q Represents the loss function of the Critic network, which measures the gap between the current Q value and the TD target; Q(z t ,a t ) represents the current Critic network state z t and action a t Estimated value of

[0216] (4) The Actor network updates its parameters by maximizing the Q value of the Critic network's output action. The formula is as follows:

[0217]

[0218] in, Represents the objective function gradient of the Actor network parameters, which is used to update the Actor parameters; Q(z,a|θ Q ) represents the value estimate of the current Critic network for any state z and action a, and the parameter is θ 4 ; represents the gradient of Q value with respect to action a, which measures the impact of action change on value; a = μ(z) means replacing a with the action output by the Actor network; Represents the gradient of the Actor network output to its own parameters; θ μ Represents the parameters of the Actor network;

[0219] (5) Soft update of the target network (Polyak averaging). The target network parameters slowly follow the changes of the main network parameters to improve the training stability. The formula is as follows:

[0220] θ Q′ ←τθ Q +(1-τθ Q′ ;

[0221] θ μ′ ←τθ μ +(1-τ)θ μ′ ;

[0222] Among them, τ∈(0,1) represents the soft update rate, which takes a value of 0.005 and is used to smooth parameter updates to prevent oscillations.

[0223] The pluggable sensing module in this embodiment has the following functions:

[0224] ① Supports various heterogeneous sensor modules and can provide feedback on task rewards to the main control module, improving collaborative perception and decision-making capabilities; heterogeneous sensor modules include image acquisition sensors, scanning area sensors, obstacle detection sensors, and pollution identification sensors;

[0225] ② Support heterogeneous task modules, flexible mounting and unmounting, and decoupling from the main control system (vehicle control); among them, heterogeneous task modules include pollution detection module, crack identification module and temperature monitoring module;

[0226] ③ Unified interface specifications, support module registration / deregistration;

[0227] ④ Output task results or events (such as "pollution detected");

[0228] ⑤ Feedback reward signal (passed to the DRL module).

[0229] Example 2:

[0230] As attached Figure 1 As shown, this embodiment provides a sewage pipe robot autonomous inspection and return control system, which is used to implement the sewage pipe robot autonomous inspection and return control method as in Example 1; the system includes:

[0231] The data dimensionality reduction module receives high-dimensional environmental and motion state data from multimodal sensors and uses dynamic principal component analysis (Dynamic PCA) to perform real-time dimensionality reduction on the high-dimensional environmental and motion state data, extracting key features of the robot's own state information and the pipeline's internal state information to construct a low-dimensional state vector.

[0232] A deep reinforcement learning control module based on liquid neural networks is used to model dynamic environmental state changes using liquid neural networks (LNNs). It combines the deep deterministic policy gradient (DDPG) algorithm to implement path planning and control, define the state space (i.e., action space), and design a multi-factor reward function to guide the robot to complete the task.

[0233] The trajectory tracing module records the robot's historical motion trajectory in the pipeline. It combines spatial change thresholds with a trajectory interpolation algorithm to compress the trajectory, retain key path feature points, and identify and store key event nodes such as pollution sources and frequent obstacles. This ensures high-precision path reconstruction and event backtracking even under limited memory resources.

[0234] The return strategy module based on graph attention network and reinforcement learning (GAT+DRL) is used to build a trajectory graph model based on the robot's historical motion trajectory when the robot needs to return. The key path nodes and trajectory compression points are defined as nodes in the graph, and the trajectory sequence and feasible paths are used as edges of the graph. The graph attention network (GAT) is combined to update the node embedding, enhance the perception ability of key areas, and plan the return path based on the reinforcement learning algorithm to improve the efficiency and safety of the return.

[0235] As attached Figure 5 As shown, the liquid neural network in this embodiment includes a state encoder, an actor network, a critic network, an actor target network, and a critic target network;

[0236] The state encoder (LNN, Liquid Neural Network) is used to process complex time series inputs and model the impact of state history on current decisions. The dynamic update formula of the state encoder is as follows:

[0237]

[0238] z t =u t ;

[0239] Among them, x t represents the low-dimensional state vector; μ t represents the internal state of the neuron; Δt represents the discrete time step; τ(x t ) represents the state-related time constant, allowing the network to respond to different inputs at different speeds; W in Represents the input weight matrix; W rec Represents the weight matrix between recursive connections (hidden layers); b represents the bias vector; f(.) represents the nonlinear activation function, and the final output z t is the encoded state, used for subsequent strategy evaluation;

[0240] The Actor network is used to output actions based on the state encoder. The formula is as follows:

[0241] a t =μ(z t |θ μ );

[0242] Among them, μ represents the policy function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action, that is, the value of linear velocity and angular velocity; θ μ Represents the learnable parameters of the Actor network;

[0243] The Critic network is used as an action value function to evaluate the quality of the current state-action pair, expressed as: Q(z t ,a t |θ Q );

[0244] Where Q(·) represents the action value function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q Represents the parameters of the Critic network, which is used to update the Actor and serve as the basis for estimating the objective function return;

[0245] The Actor target network and the Critic target network are used to construct stable target values. Specifically, the slowly updated Actor target network and Critic target network are introduced through the deep deterministic policy gradient algorithm, corresponding to the copies of the Actor and Critic respectively, and are defined as: μ′≈μ,Q′≈Q;

[0246] Among them, μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; Q′ represents the Critic target network.

[0247] The training process of the liquid neural network in this embodiment is as follows:

[0248] The agent interacts with the environment: During each interaction, the agent selects an action based on its current state, and the environment feeds back the new state and reward. The "state-action-reward-new state" sequence is recorded as data for subsequent learning.

[0249] The TD target is used to update the Critic network and represents "current reward + estimate of future reward". The TD target calculation formula is as follows:

[0250] y t =r t +γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ );

[0251] Among them, y tRepresents the TD target value of this step, which is used to train the Critic network; r t represents the immediate reward obtained at the current moment t; γ represents the discount factor, which ranges from 0 to 1 and measures the importance of future rewards; Q′ represents the Critic target network, which is used to estimate the action value of the next state; z t+1 Represents the state characteristics of the next moment; μ′ represents the Actor target network, which is used to give the optimal action in the next state; θ μ′ Represents the parameters of the Actor target network; θ Q′ Represents the parameters of the Critic target network;

[0252] Minimize the loss: L Q =(Q(z t ,a t )-y t ) 2 ;

[0253] Among them, L Q Represents the loss function of the Critic network, which measures the gap between the current Q value and the TD target; Q(z t ,a t ) represents the current Critic network state z t and action a t Estimated value of

[0254] The Actor network updates its parameters by maximizing the Q value of the Critic network's output action. The formula is as follows:

[0255]

[0256] in, Represents the objective function gradient of the Actor network parameters, which is used to update the Actor parameters; Q(z,a|θ Q ) represents the value estimate of the current Critic network for any state z and action a, and the parameter is θ Q ; represents the gradient of Q value with respect to action a, which measures the impact of action change on value; a = μ(z) means replacing a with the action output by the Actor network; Represents the gradient of the Actor network output to its own parameters; θ μ Represents the parameters of the Actor network;

[0257] Soft update of the target network (Polyak averaging) allows the target network parameters to slowly follow the changes in the main network parameters to improve training stability. The formula is as follows:

[0258] θ Q′ ←τθ Q+(1-τθ Q′ ;

[0259] θ μ′ ←τθ μ +(1-τ)θ μ′ ;

[0260] Among them, τ∈(0,1) represents the soft update rate, which takes a value of 0.005 and is used to smooth parameter updates to prevent oscillations.

[0261] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A sewage pipe robot autonomous inspection and return control method, characterized in that: The method is as follows: Data dimensionality reduction: Receive high-dimensional environment and motion state data from multimodal sensors, use dynamic principal component analysis to perform real-time dimensionality reduction on the high-dimensional environment and motion state data, extract key features of the robot's own state information and the pipeline's internal state information, and construct a low-dimensional state vector; Deep reinforcement learning control based on liquid neural networks: This approach uses liquid neural networks to model dynamic environmental state changes, combines them with deep deterministic policy gradient algorithms to implement path planning and control, and defines the state space (i.e., action space). It also designs a multi-factor reward function to guide the robot in completing tasks. Trajectory tracing: Records the robot's historical motion trajectory in the pipeline, combines spatial change thresholds with trajectory interpolation algorithms to compress the trajectory, retains key path feature points, and identifies and stores key event nodes, ensuring high-precision path reconstruction and event backtracking even under limited memory resources. Return strategy based on graph attention network and reinforcement learning: When the robot needs to return, a trajectory graph model is constructed based on the robot's historical motion trajectory. Key path nodes and trajectory compression points are defined as nodes in the graph, and trajectory sequence and feasible paths are used as edges of the graph. The graph attention network is combined to update the node embedding, enhance the perception ability of key areas, and plan the return path based on the reinforcement learning algorithm.

2. The sewage pipe robot autonomous inspection and return control method according to claim 1 is characterized in that: The data dimensionality reduction is as follows: Collect the robot's own state information and the internal state information of the pipeline; the robot's own state information includes the current linear velocity, current angular velocity, posture information, three-axis acceleration, three-axis angular acceleration, task completion status, and the history of the previous steps; the internal state information of the pipeline includes the distance to obstacles in each direction, local slope, pipeline curvature, and pass probability; Construct the original state vector s t , specifically: s t =[robot status, pipeline internal environment status]; The original state vector s is transformed into t Perform dimensionality reduction to obtain the low-dimensional state vector x t , the formula is: in, represents the mean of the original state vector; Represents the matrix used to project the original state vector, and the original state vector is projected into a k-dimensional space through the covariance matrix.

3. The sewage pipe robot autonomous inspection and return control method according to claim 1 is characterized in that: The details of deep reinforcement learning control based on liquid neural networks are as follows: Define the state space: The state space is defined as a low-dimensional state vector x t ; Define the action space: The linear velocity and angular velocity range of the current action output is defined as the action space, expressed as a t =[v t ,ω t ]; where v represents linear velocity, v∈[-2,2], in meters per second; ω represents angular velocity, ω∈[-1,1], in rad per second; Design a reward function: This integrates multiple factors to guide the robot to move forward efficiently, avoid obstacles, and complete exploration tasks. The formula is as follows: R t =W1×r forward +W2×r safe +W3×r task +W4×r smooth ; Among them, r forward Represents the reward value for encouraging movement in the target direction; r safe Indicates the reward value for maintaining a safe distance and avoiding obstacles; r task represents the task completion degree of the pluggable perception module, that is, the target task completion rate; r smooth represents the oscillation penalty. A larger reward value indicates a more dramatic change, i.e., severe "jitter" that requires a penalty. W1, W2, W3, and W4 are the weights of the reward value for encouraging movement toward the target, the weights of the reward value for maintaining a safe distance and avoiding obstacles, the weight of the task completion of the pluggable perception module, and the weight of the oscillation penalty, respectively. Normalization is required before calculating the reward value for encouraging movement toward the target, the weights of the reward value for maintaining a safe distance and avoiding obstacles, the weight of the task completion of the pluggable perception module, and the weight of the oscillation penalty. The specific calculations are as follows: r forward =V t ×cos(θ t ); Among them, V t Indicates the current speed of the robot; θ t Indicates the angle between the robot's current moving direction and the target direction; cos(θ t ) indicates whether the movement is towards the target. The closer to 1, the better. If the movement is towards the target, the reward is positive; if the movement is in the opposite direction, the reward is negative. Among them, d t Indicates the distance between the robot and the nearest obstacle; d safe Indicates the set minimum safety distance threshold. If you are too close to an obstacle, you will receive a negative reward. Among them, n t Indicates the number of tasks actually completed in the time step; n total Indicates the number of tasks expected to be completed this time; r smooth =(a t -a t-1 ) 2 Among them, a t Indicates the action at the current time step; a t-1 Represents the action at the previous time step.

4. The sewage pipe robot autonomous inspection and return control method according to claim 1 is characterized in that: The liquid neural network includes a state encoder, an actor network, a critic network, an actor target network, and a critic target network; The state encoder is used to process complex time series inputs and model the impact of state history on current decisions. The dynamic update formula of the state encoder is as follows: With t =u t ; Among them, x t represents the low-dimensional state vector; μ t represents the internal state of the neuron; Δt represents the discrete time step; τ(x t ) represents the state-related time constant, allowing the network to respond to different inputs at different speeds; W in Represents the input weight matrix; W rec Represents the weight matrix between recursive connections; b represents the bias vector; f(.) represents the nonlinear activation function, and the final output z t is the encoded state, used for subsequent strategy evaluation; The Actor network is used to output actions based on the state encoder. The formula is as follows: a t =μ(z t |θ μ ); Among them, μ represents the policy function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action, that is, the value of linear velocity and angular velocity; θ μ Represents the learnable parameters of the Actor network; The Critic network is used as an action value function to evaluate the quality of the current state-action pair, expressed as: Q(z t ,a t |θ Q ); Where Q(·) represents the action value function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q Represents the parameters of the Critic network, which is used to update the Actor and serve as the basis for estimating the objective function return; The Actor target network and the Critic target network are used to construct stable target values. Specifically, the slowly updated Actor target network and Critic target network are introduced through the deep deterministic policy gradient algorithm, corresponding to the copies of the Actor and Critic respectively, and are defined as: μ′≈μ,Q′≈Q; Among them, μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; Q′ represents the Critic target network.

5. The sewage pipe robot autonomous inspection and return control method according to claim 1 or 4, characterized in that: The training process of the liquid neural network is as follows: The agent interacts with the environment: During each interaction, the agent selects an action based on its current state, and the environment provides feedback on the new state and reward. The "state-action-reward-new state" sequence is recorded as data for subsequent learning. The TD target is used to update the Critic network and represents "current reward + estimate of future reward". The TD target calculation formula is as follows: y t =r t +γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ ); Among them, y t Represents the TD target value of this step, which is used to train the Critic network; r t represents the immediate reward obtained at the current moment t; γ represents the discount factor, which ranges from 0 to 1 and measures the importance of future rewards; Q′ represents the Critic target network, which is used to estimate the action value of the next state; z t+1 Represents the state characteristics of the next moment; μ′ represents the Actor target network, which is used to give the optimal action in the next state; θ μ′ Represents the parameters of the Actor target network; θ Q′ Represents the parameters of the Critic target network; Minimize the loss: L Q =(Q(z t ,a t )-y t ) 2 ; Among them, L q Represents the loss function of the Critic network, which measures the gap between the current Q value and the TD target; Q(z t ,a t ) represents the current Critic network state z t and action a t Estimated value of The Actor network updates its parameters by maximizing the Q value of the Critic network's output action. The formula is as follows: in, Represents the objective function gradient of the Actor network parameters, which is used to update the Actor parameters; Q(z,a|θ Q ) represents the value estimate of the current Critic network for any state z and action a, and the parameter is θ Q ; represents the gradient of Q value with respect to action a, which measures the impact of action change on value; a = μ(z) means replacing a with the action output by the Actor network; Represents the gradient of the Actor network output to its own parameters; θ μ Represents the parameters of the Actor network; Soft update the target network. The target network parameters slowly follow the changes of the main network parameters to improve training stability. The formula is as follows: i Q′ ←tth Q +(1-tθ QQ ; i μQ ←tth μ +(1-τ)θ μ′ ; Among them, τ∈(0,1) represents the soft update rate, which is used to smooth parameter updates to prevent oscillations.

6. The sewage pipe robot autonomous inspection and return control method according to claim 1 is characterized in that: The traceability details are as follows: Real-time recording and compressed storage of trajectories: Assume that the position of the robot at time t is X t , and its trajectory sequence is expressed as: τ={X1,X2,…,X n }; In order to reduce the memory burden, linear interpolation combined with key point preservation algorithm is used to compress the trajectory. The formula is: Among them, δ represents the spatial variation threshold, and the compression function keeps the path accuracy error no more than the preset value; Key node marking and storage: The robot uses sensors to identify the pollution source point P 污染源 ={p 污染源i }、Obstacle source hotspot P 障碍源 ={p 障碍源i }, the key node set is defined as: k = P 污染源 ∪P 障碍源 The key node set will be stored first and will not be affected by trajectory compression, so as to provide reference for return planning and replay after the inspection mission is completed; Trajectory reconstruction capability: Using compressed trajectories and key node sets to perform approximate trajectory restoration, we can achieve a near-true path tracing capability, which is expressed as:

7. The sewage pipe robot autonomous inspection and return control method according to claim 1 is characterized in that: The return strategy based on graph attention network and reinforcement learning is as follows: Environment graph construction: Construct pipeline graph model G = (V, E); where the node set V contains trajectory compression points All positions and key nodes k in The construction method ensures that key nodes are perceived first, improving the interpretability and safety of the return path; the edge set E consists of trajectory sequences or feasible paths, and is assigned obstacle risk and distance weights; Graph attention embedding extraction: The graph attention mechanism is used to update the node embedding, and each node embedding h i The transformation formula is as follows: Among them, α ij represents the attention weight of node j to node i; W represents the linear transformation matrix of the graph attention layer, which is used to project the original features into the new feature space; h i represents the original feature vector of node i; a represents the parameter vector of the attention mechanism, which is used to score the correlation between node pairs; N(i) represents the set of neighbor nodes of node i; the node feature update formula is: i ′=σ(∑ j∈N(i) α ij Wh j ); where h i ′ represents the updated node features, integrating neighbor node information; σ represents the activation function; W represents the learnable weight matrix for feature transformation; h j Represents the current feature vector of neighbor node j; Reinforcement learning modeling, as follows: State space design: state vector s t By the current position node h i ′ and the environmental state, the formula is as follows: Among them, h i i represents the embedded representation of the current node after GAT encoding; θ t Indicates the current heading angle; e t Indicates the current remaining energy; Indicates the distance from the current position to the return target point; rask t Indicates the risk value of the current location or path; Action space design: The linear velocity and angular velocity range of the current action output is defined as the action space, expressed as a t =[v t ,ω t ]; where v represents linear velocity, v∈[-2,2], in meters per second; ω represents angular velocity, ω∈[-1,1], in rad per second; The reward function is designed with the following formula: Among them: i represents the reward weight; Indicates the distance from the current position to the return target point; rask t Indicates the risk value of the current location or path; collision t Indicates whether a collision has occurred; isdone indicates whether the return target point has been reached; Policy network design: including state encoder, actor network, critic network, actor target network and critic target network.

8. The sewage pipe robot autonomous inspection and return control system according to claim 3 is characterized in that: The pluggable sensor module has the following functions: ① Supports various heterogeneous sensor modules and can provide feedback on task rewards to the main control module, improving collaborative perception and decision-making capabilities; heterogeneous sensor modules include image acquisition sensors, scanning area sensors, obstacle detection sensors, and pollution identification sensors; ②Support heterogeneous task modules, flexible mounting and unmounting, and decoupling from the main control system; the heterogeneous task modules include pollution detection module, crack identification module, and temperature monitoring module; ③ Unified interface specifications, support module registration / deregistration; ④ Output task results or events; ⑤Feedback reward signal.

9. A sewage pipe robot autonomous inspection and return control system, characterized in that: The system is used to implement the autonomous inspection and return control method of the sewage pipe robot according to any one of claims 1 to 8; the system includes: The data dimensionality reduction module receives high-dimensional environment and motion state data from multimodal sensors, uses dynamic principal component analysis to perform real-time dimensionality reduction on the high-dimensional environment and motion state data, extracts key features of the robot's own state information and the pipeline's internal state information, and constructs a low-dimensional state vector. A deep reinforcement learning control module based on liquid neural networks is used to model dynamic environmental state changes using liquid neural networks. It combines deep deterministic policy gradient algorithms to implement path planning and control, define the state space (i.e., action space), and design a multi-factor reward function to guide the robot to complete tasks. The trajectory tracing module is used to record the robot's historical motion trajectory in the pipeline. It combines spatial change thresholds with trajectory interpolation algorithms to compress the trajectory, retain key path feature points, and identify and store key event nodes, ensuring high-precision path reconstruction and event backtracking under limited memory resources. The return strategy module based on graph attention network and reinforcement learning is used to build a trajectory graph model based on the robot's historical motion trajectory when the robot needs to return. The key path nodes and trajectory compression points are defined as nodes in the graph, and the trajectory sequence and feasible paths are used as edges of the graph. The graph attention network is combined to update the node embedding, enhance the perception ability of key areas, and plan the return path based on the reinforcement learning algorithm.

10. The sewage pipe robot autonomous inspection and return control system according to claim 9, characterized in that: The liquid neural network includes a state encoder, an actor network, a critic network, an actor target network, and a critic target network; The state encoder is used to process complex time series inputs and model the impact of state history on current decisions. The dynamic update formula of the state encoder is as follows: With t =u t ; Among them, x t represents the low-dimensional state vector; μ t represents the internal state of the neuron; Δt represents the discrete time step; τ(x t ) represents the state-related time constant, allowing the network to respond to different inputs at different speeds; W in Represents the input weight matrix; W rec Represents the weight matrix between recursive connections; b represents the bias vector; f(.) represents the nonlinear activation function, and the final output z t is the encoded state, used for subsequent strategy evaluation; The Actor network is used to output actions based on the state encoder. The formula is as follows: a t =μ(z t |θ μ ); Among them, μ represents the policy function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action, that is, the value of linear velocity and angular velocity; θ μ Represents the learnable parameters of the Actor network; The Critic network is used as an action value function to evaluate the quality of the current state-action pair, expressed as: Q(z t ,a t |θ Q ); Where Q(·) represents the action value function; z t represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q Represents the parameters of the Critic network, which is used to update the Actor and serve as the basis for estimating the objective function return; The Actor target network and the Critic target network are used to construct stable target values. Specifically, the slowly updated Actor target network and Critic target network are introduced through the deep deterministic policy gradient algorithm, corresponding to the copies of the Actor and Critic respectively, and are defined as: μ′≈μ,Q′≈Q; Among them, μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; Q′ represents the Critic target network; The training process of the liquid neural network is as follows: The agent interacts with the environment: During each interaction, the agent selects an action based on its current state, and the environment provides feedback on the new state and reward. The "state-action-reward-new state" sequence is recorded as data for subsequent learning. The TD target is used to update the Critic network and represents "current reward + estimate of future reward". The TD target calculation formula is as follows: y t =r t +γQ′(z t+1 ,μ′(z t+1 |θ μ4 )|θ 4′ ); Among them, y t Represents the TD target value of this step, which is used to train the Critic network; r t represents the immediate reward obtained at the current moment t; γ represents the discount factor, which ranges from 0 to 1 and measures the importance of future rewards; Q′ represents the Critic target network, which is used to estimate the action value of the next state; z t+1 Represents the state characteristics of the next moment; μ′ represents the Actor target network, which is used to give the optimal action in the next state; θ μ′ Represents the parameters of the Actor target network; θ Q′ Represents the parameters of the Critic target network; Minimize the loss: L Q =(Q(z t ,a t )-y t ) 2 ; Among them, L Q Represents the loss function of the Critic network, which measures the gap between the current Q value and the TD target; Q(z t ,a t ) represents the current Critic network state z t and action a t Estimated value of The Actor network updates its parameters by maximizing the Q value of the Critic network's output action. The formula is as follows: in, Represents the objective function gradient of the Actor network parameters, which is used to update the Actor parameters; Q(z,a|θ Q ) represents the value estimate of the current Critic network for any state z and action a, and the parameter is θ Q ; represents the gradient of Q value with respect to action a, which measures the impact of action change on value; a = μ(z) means replacing a with the action output by the Actor network; Represents the gradient of the Actor network output to its own parameters; θ μ Represents the parameters of the Actor network; Soft update the target network. The target network parameters slowly follow the changes of the main network parameters to improve training stability. The formula is as follows: i Q′ ←tth Q +(1-tθ Q′ ; i μ′ ←tth μ +(1-τ)θ μ′ ; Among them, τ∈(0,1) represents the soft update rate, which is used to smooth parameter updates to prevent oscillations.

Images