Sewer pipeline robot autonomous inspection and return control method and system

Abstract

The application discloses a sewer pipeline robot autonomous inspection and return control method and system, belongs to the technical field of intelligent pipeline detection and robot control, and aims to solve the technical problem of how to realize stable, reliable and continuous silt exploration tasks in a complex sewer pipeline environment and have safe self-rescue in an abnormal state. The technical scheme is as follows: data dimension reduction: receiving high-dimensional environment and motion state data from multi-modal sensors, adopting a dynamic principal component analysis method to perform real-time dimension reduction on the high-dimensional environment and motion state data, extracting key features of the robot state information and the pipeline internal state information, and constructing a low-dimensional state vector; deep reinforcement learning control based on a liquid-state neural network: modeling dynamic environment state changes by using a liquid-state neural network, and realizing path planning and control by combining a deep deterministic policy gradient algorithm; trajectory tracing; and return strategy based on a graph attention network and reinforcement learning.

Description

Technical Field

[0001] This invention relates to the field of intelligent pipeline inspection and robot control technology, specifically a method and system for autonomous inspection and return control of sewage pipeline robots. Background Technology

[0002] Currently, the inspection and blockage detection of sewage pipelines mainly rely on manual entry or remotely controlled robotic systems. This not only suffers from low operational efficiency and high operational risks, but also suffers from severely limited control precision and task completion capabilities when dealing with long-distance, winding, and complex underground pipeline networks. Especially in extreme environments such as high pollution, low light, and frequent water level changes, existing technologies cannot achieve reliable continuous operation.

[0003] With the development of deep reinforcement learning and mobile robotics, autonomous decision-making and intelligent control have become research hotspots. Although existing methods have attempted to introduce deep reinforcement learning into pipeline inspection tasks, traditional neural network models have poor adaptability to dynamic environments, easily leading to decision delays or even control failures when faced with high-dimensional state inputs and uncertain environmental dynamics. Furthermore, sewage pipeline robots are typically deployed on resource-constrained embedded platforms with limited computing resources and power supplies, making it difficult to support the high-frequency computation of large-scale neural networks. Meanwhile, robots often face challenges such as complex return paths, loss of historical trajectories, and sensor drift during task execution, making safe return difficult in situations such as insufficient power or positioning interruptions.

[0004] Therefore, how to achieve stable, reliable, and continuous sludge detection in complex sewage pipeline environments, and how to ensure safe self-rescue in abnormal conditions, are urgent technical problems that need to be solved. Summary of the Invention

[0005] The technical objective of this invention is to provide a method and system for autonomous inspection and return control of sewage pipeline robots, in order to solve the problem of how to achieve stable, reliable, and continuous siltation detection in complex sewage pipeline environments, and to have the ability to safely self-rescue in abnormal situations.

[0006] The technical objective of this invention is achieved as follows: a method for autonomous inspection and return control of a sewage pipeline robot, the specific method of which is as follows:

[0007] Data dimensionality reduction: Receive high-dimensional environmental and motion state data from multimodal sensors, and use Dynamic Principal Component Analysis (Dynamic PCA) to perform real-time dimensionality reduction on the high-dimensional environmental and motion state data. Extract key features of the robot's own state information and the internal state information of the pipeline to construct a low-dimensional state vector.

[0008] Deep reinforcement learning control based on liquid neural network: The liquid neural network (LNN) is used to model the dynamic environmental state changes, and the deep deterministic policy gradient (DDPG) algorithm is combined to realize path planning and control. The state space is defined as the action space, and a multi-factor reward function is designed to guide the robot to complete the task.

[0009] Trajectory tracing: Record the robot's historical movement trajectory in the pipeline, combine spatial change threshold and trajectory interpolation algorithm to compress the trajectory, retain key path feature points, and identify and store key event nodes such as pollution sources and frequent obstacles to ensure that high-precision path reconstruction and event backtracking can still be achieved under limited memory resources.

[0010] The Return-to-Home Strategy Based on Graph Attention Network and Reinforcement Learning (GAT+DRL) is as follows: When the robot needs to return to its home position, a trajectory graph model is constructed based on the robot's historical trajectory. Key path nodes and trajectory compression points are defined as nodes in the graph, and the trajectory sequence and feasible paths are defined as edges of the graph. The node embedding is updated by combining the Graph Attention Network (GAT) to enhance the perception of key areas. The return-to-home path is planned based on the reinforcement learning algorithm to improve the efficiency and safety of the return-to-home process.

[0011] As a preferred approach, data dimensionality reduction is performed as follows:

[0012] Collect robot's own state information and the state information inside the pipeline; the robot's own state information includes current linear velocity, current angular velocity, pose information, three-axis acceleration, three-axis angular acceleration, task completion status, and previous historical actions; the state information inside the pipeline includes obstacle distances in each direction, local slope, pipeline curvature, and probability of passage.

[0013] Construct the original state vector s t Specifically: s t = [Robot's own state, pipe's internal environment state];

[0014] The original state vector s is obtained through dynamic principal component analysis. t Perform dimensionality reduction to obtain the low-dimensional state vector x t The formula is: in, This represents the mean of the original state vector; This represents the matrix used to project the original state vector into a k-dimensional space through the covariance matrix.

[0015] As a preferred embodiment, the deep reinforcement learning control based on liquid neural networks is as follows:

[0016] Define the state space: Define the state space as a low-dimensional state vector x. t ;

[0017] Define the motion space: Define the range of linear and angular velocities output by the current action as the motion space, denoted as a. t =[v t ,ω t Where v represents linear velocity, v∈[-2,2], in meters per second; ω represents angular velocity, ω∈[-1,1], in rad / second;

[0018] Design a reward function: This function integrates multiple factors to guide the robot to move efficiently forward, avoid obstacles, and complete its exploration task. The formula is as follows:

[0019] R t =W1×r forward +W2×r safe +W3×r task +W4×r smooth ;

[0020] Where, r forward The reward value represents the incentive to move in the target direction; r safe This represents the reward value for maintaining a safe distance and avoiding obstacles; r task This represents the task completion rate of the pluggable sensing module, i.e., the target task completion rate; r smooth The reward value represents the oscillation penalty; a larger reward value indicates more drastic changes, i.e., severe "jitter," requiring penalty. W1, W2, W3, and W4 are the weights of the reward value for encouraging movement in the target direction, the reward value for maintaining a safe distance, the reward value for obstacle avoidance, the weight of the task completion rate of the pluggable sensing module, and the weight of the oscillation penalty, respectively. Normalization processing is performed before calculating the reward value for encouraging movement in the target direction, the reward value for maintaining a safe distance, the reward value for obstacle avoidance, the task completion rate of the pluggable sensing module, and the oscillation penalty, as detailed below:

[0021] r forward =V t ×cos(θ t );

[0022] Among them, V t θ represents the robot's current speed. t Cos(θ) represents the angle between the robot's current direction of motion and the target direction. t The ) indicates whether the movement is toward the target. The closer to 1, the better. If the movement is toward the target, the reward is positive; if it is in the opposite direction, the reward is negative.

[0023]

[0024] Where, d t Indicates the distance between the robot and the nearest obstacle; d safeThis indicates the set minimum safe distance threshold; if you get too close to an obstacle, you will receive a negative reward.

[0025]

[0026] Where, n t n represents the number of tasks actually completed at each time step. total This indicates the number of tasks expected to be completed this time;

[0027] r smooth =(a t -a t-1 ) 2 ;

[0028] Among them, a t This indicates the action at the current time step; a t-1 It indicates the action at the previous time step.

[0029] Preferably, the liquid neural network includes a state encoder, an Actor network, a Critic network, an Actor target network, and a Critic target network;

[0030] A Liquid Neural Network (LNN) is used to process complex temporal inputs and model the impact of historical states on the current decision. The dynamic update formula for the LNN is as follows:

[0031]

[0032] z t =u t ;

[0033] Where, x t μ represents a low-dimensional state vector. t Represents the internal state of a neuron; Δt represents the time elapsed since the walk; τ(x) represents the time elapsed since the walk. t ) represents the state-dependent time constant, allowing the network to respond to different inputs at different speeds; W in W represents the input weight matrix; rec The weight matrix represents the recursive connections (hidden layers); b represents the bias vector; f(.) represents the non-linear activation function; and the final output z... t This is the encoded state, used for subsequent policy evaluation;

[0034] Actor networks are used to output actions based on state encoders, as shown in the following formula:

[0035] a t =μ(z) t |θ μ );

[0036] Where μ represents the policy function; z t This represents the temporal features extracted by the state encoder; a t This indicates the currently selected action, i.e., the values ​​of linear velocity and angular velocity; θ μ Represents the learnable parameters of the Actor network;

[0037] The Critic network, used as an action-value function, evaluates the quality of the current state-action pair, denoted as: Q(z) t ,a t |θ Q );

[0038] Where Q(·) represents the action-value function; z t This represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q The parameters of the Critic network are used to update the Actors and serve as an estimate of the objective function's return.

[0039] The Actor and Critic target networks are used to construct stable target values. Specifically, a slowly updated Actor and Critic target network is introduced through a deep deterministic policy gradient algorithm. These networks correspond to copies of the Actor and Critic, respectively, and are defined as: μ′≈μ, Q′≈Q.

[0040] Where μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; and Q′ represents the Critic target network.

[0041] More preferably, the training process of the liquid neural network is as follows:

[0042] The agent interacts with the environment: each time the agent interacts, it selects an action based on the current state, and the environment provides feedback on the new state and reward. The "state-action-reward-new state" is recorded as data for subsequent learning.

[0043] The TD objective is used to update the Critic network and represents "current reward + estimate of future reward". The formula for calculating the TD objective is as follows:

[0044] y t =r t +γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ );

[0045] Among them, y t This represents the TD target value for this step, used to train the Critic network; rt The z-axis represents the immediate reward obtained at the current time t; γ represents the discount factor, which takes a value between 0 and 1 and measures the importance of future rewards; Q′ represents the Critic target network, used to estimate the action value of the next state; z t+1 The state characteristics at the next time step are represented by μ′; μ′ represents the Actor target network, used to give the optimal action in the next state; θ μ′ θ represents the parameters of the Actor target network. Q′ These represent the parameters of the Critic target network;

[0046] The minimum loss is: L Q =(Q(z) t ,a t )-y t ) 2 ;

[0047] Among them, L Q The loss function of the Critic network is Q(z), which measures the difference between the current Q value and the TD target. t ,a t ) represents the current state z of the Critic network. t and action a t Value estimation;

[0048] The Actor network updates its parameters by maximizing the Q-value of the Critic network's output action, as shown in the following formula:

[0049]

[0050] in, The gradient of the objective function representing the Actor network parameters is used to update the Actor parameters; Q(z,a|θ) Q ) represents the current Critic network's value estimate for any state z and action a, with parameter θ. Q ; This represents the gradient of the Q-value with respect to action a, measuring the impact of the action change on the value; a = μ(z) means replacing a with the action output by the Actor network; θ represents the gradient of the Actor network output with respect to its own parameters. μ Represents the parameters of the Actor network;

[0051] Soft update of the target network (Polyak averaging) allows the target network parameters to slowly follow the changes in the main network parameters, improving training stability. The formula is as follows:

[0052] θ Q′ ←τθ Q +(1-τθ Q′ ;

[0053] θ μ′ ←τθ μ +(1-τ)θ μ′ ;

[0054] Where τ∈(0,1) represents the soft update rate, which takes the value 0.005 and is used to smooth parameter updates and prevent oscillations.

[0055] As a preferred option, the trajectory tracing is as follows:

[0056] Real-time trajectory recording and compressed storage: Let the robot's position at time t be X. t Its trajectory sequence is represented as: τ={X1,X2,…,X n};

[0057] To reduce memory load, linear interpolation combined with a keypoint-preserving algorithm is used for trajectory compression, as shown in the formula: Where δ represents the spatial variation threshold, the compression function keeps the path accuracy error within a preset value;

[0058] Key node marking and storage: The robot identifies pollution source point P through sensors. 污染源 ={p 污染源i}、Barrier source hotspot P 障碍源 ={p 障碍源i The key events and key node set are defined as: k = P 污染源 ∪P 障碍源 The key node set will be stored first and will not be affected by trajectory compression, for reference in the return planning and review after the patrol mission is completed;

[0059] Trajectory reconstruction capability: By utilizing compressed trajectories and key node sets, approximate trajectory reconstruction is performed to achieve near-realistic path backtracking capability, represented as:

[0060] As a preferred approach, the return-to-home strategy based on graph attention networks and reinforcement learning is as follows:

[0061] Environment Graph Construction: Construct a pipeline graph model G = (V, E); where the node set V contains trajectory compression points. All positions and key node k; using The construction method ensures that key nodes are perceived first, improving the interpretability and safety of the return path; the edge set E consists of trajectory order or feasible paths, and is assigned obstacle risk and distance weights.

[0062] Graph attention embedding extraction: LeakyReLU (Leaky Rectified Linear Unit) is a commonly used neural network activation function, expressed as: Where α is a constant with a value of 0.01; a graph attention mechanism is used to update the node embeddings, and each node embedding h... i The transformation formula is as follows:

[0063]

[0064] Where, α ij Let represent the attention weight of node j towards node i; W represents the linear transformation matrix of the graph attention layer, used to project the original features into a new feature space; h i Let represent the original feature vector of node i; 'a' represents the parameter vector of the attention mechanism, used to score the correlation between node pairs; N(i) represents the set of neighboring nodes of node i; the node feature update formula is: h i i=σ(∑ j∈N(i) α ij Wh j ); where h i ′ represents the updated node features, incorporating neighbor node information; σ represents the activation function; W represents the learnable weight matrix used for feature transformation; h j This represents the current feature vector of neighbor node j;

[0065] Reinforcement learning modeling, specifically as follows:

[0066] State-space design: State vector s t From the current position node h i i is composed of the environment state, as shown in the following formula:

[0067]

[0068] Among them, h i ' represents the embedded representation of the current node after GAT encoding; θ t Indicates the current orientation angle; e t This represents the current remaining energy (scalar). This represents the graph distance from the current location to the return target point; rask t Indicates the risk value of the current location or path;

[0069] Motion space design: The range of linear and angular velocities of the current motion output is defined as the motion space, denoted as a. t =[v t ,ω t Where v represents linear velocity, v∈[-2,2], in meters per second; ω represents angular velocity, ω∈[-1,1], in rad / second;

[0070] The reward function is designed as follows:

[0071]

[0072] Where; λ i Indicates the reward weight; This represents the graph distance from the current location to the return target point; rask t Indicates the risk value of the current location or path; collision t Indicates whether a collision has occurred; isdone indicates whether the return-to-base destination has been reached;

[0073] Policy network design includes state encoder, Actor network, Critic network, Actor target network, and Critic target network.

[0074] More preferably, the pluggable sensing module has the following functions:

[0075] ① It supports various heterogeneous sensor modules and can provide task rewards to the main control module to improve collaborative perception and decision-making capabilities; among them, heterogeneous sensor modules include image acquisition sensors, scanning area sensors, obstacle detection sensors and pollution recognition sensors;

[0076] ② Supports heterogeneous task modules, which can be flexibly mounted and unmounted and decoupled from the main control system (vehicle control); among which, the heterogeneous task modules include pollution detection module, crack identification module and temperature monitoring module;

[0077] ③ Unified interface specifications, supporting module registration / deregistration;

[0078] ④ Output task results or events (such as "contamination detected");

[0079] ⑤ Feedback reward signal (transmitted to the DRL module).

[0080] An autonomous inspection and return control system for a sewage pipeline robot, the system being used to implement the aforementioned autonomous inspection and return control method for a sewage pipeline robot; the system includes:

[0081] The data dimensionality reduction module is used to receive high-dimensional environmental and motion state data from multimodal sensors, and uses the Dynamic Principal Component Analysis (Dynamic PCA) method to perform real-time dimensionality reduction on the high-dimensional environmental and motion state data, extract key features of the robot's own state information and the internal state information of the pipeline, and construct a low-dimensional state vector.

[0082] The deep reinforcement learning control module based on liquid neural network is used to model dynamic environmental state changes using liquid neural network (LNN), combine deep deterministic policy gradient (DDPG) algorithm to realize path planning and control, define the state space as action space, and design multi-factor reward function to guide the robot to complete the task.

[0083] The trajectory tracing module is used to record the robot's historical movement trajectory in the pipeline. It combines spatial change threshold and trajectory interpolation algorithm to compress the trajectory, retain key path feature points, and identify and store key event nodes such as pollution sources and frequent obstacles, so as to ensure that high-precision path reconstruction and event backtracking can still be achieved under the condition of limited memory resources.

[0084] The Return-to-Home Strategy (GAT+DRL) module, based on Graph Attention Network and Reinforcement Learning, is used to construct a trajectory graph model based on the robot's historical motion trajectory when the robot needs to return to home. Key path nodes and trajectory compression points are defined as nodes in the graph, and trajectory order and feasible paths are defined as edges of the graph. The node embedding is updated by combining Graph Attention Network (GAT) to enhance the perception of key areas, and the return-to-home path is planned based on reinforcement learning algorithm to improve return-to-home efficiency and safety.

[0085] Preferably, the liquid neural network includes a state encoder, an Actor network, a Critic network, an Actor target network, and a Critic target network;

[0086] A Liquid Neural Network (LNN) is used to process complex temporal inputs and model the impact of historical states on the current decision. The dynamic update formula for the LNN is as follows:

[0087]

[0088] z t =u t ;

[0089] Where, x t μ represents a low-dimensional state vector. t Represents the internal state of a neuron; Δt represents the time elapsed since the walk; τ(x) represents the time elapsed since the walk. t ) represents the state-dependent time constant, allowing the network to respond to different inputs at different speeds; W in W represents the input weight matrix; rec The weight matrix represents the recursive connections (hidden layers); b represents the bias vector; f(.) represents the non-linear activation function; and the final output z... t This is the encoded state, used for subsequent policy evaluation;

[0090] Actor networks are used to output actions based on state encoders, as shown in the following formula:

[0091] a t =μ(z) t |θ μ );

[0092] Where μ represents the policy function; z t This represents the temporal features extracted by the state encoder; a t This indicates the currently selected action, i.e., the values ​​of linear velocity and angular velocity; θ μ Represents the learnable parameters of the Actor network;

[0093] The Critic network, used as an action-value function, evaluates the quality of the current state-action pair, denoted as: Q(z) t ,a t |θ Q );

[0094] Where Q(·) represents the action-value function; z t This represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q The parameters of the Critic network are used to update the Actors and serve as an estimate of the objective function's return.

[0095] The Actor and Critic target networks are used to construct stable target values. Specifically, they are introduced using a deep deterministic policy gradient algorithm, resulting in slowly updated Actor and Critic target networks, which correspond to copies of the Actor and Critic, respectively, defined as: μ′≈μ,Q ′ ≈Q;

[0096] Where μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; Q′ represents the Critic target network;

[0097] The training process of a liquid neural network is as follows:

[0098] The agent interacts with the environment: each time the agent interacts, it selects an action based on the current state, and the environment provides feedback on the new state and reward. The "state-action-reward-new state" is recorded as data for subsequent learning.

[0099] The TD objective is used to update the Critic network and represents "current reward + estimate of future reward". The formula for calculating the TD objective is as follows:

[0100] y t =r t+γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ );

[0101] Among them, y t This represents the TD target value for this step, used to train the Critic network; r t The z-axis represents the immediate reward obtained at the current time t; γ represents the discount factor, which takes a value between 0 and 1 and measures the importance of future rewards; Q′ represents the Critic target network, used to estimate the action value of the next state; z t+1 The state characteristics at the next time step are represented by μ′; μ′ represents the Actor target network, used to give the optimal action in the next state; θ μ′ θ represents the parameters of the Actor target network. Q′ These represent the parameters of the Critic target network;

[0102] The minimum loss is: L Q =(Q(z) t ,a t )-y t ) 2 ;

[0103] Among them, L Q The loss function of the Critic network is Q(z), which measures the difference between the current Q value and the TD target. t ,a t ) represents the current state z of the Critic network. t and action a t Value estimation;

[0104] The Actor network updates its parameters by maximizing the Q-value of the Critic network's output action, as shown in the following formula:

[0105]

[0106] in, The gradient of the objective function representing the Actor network parameters is used to update the Actor parameters; Q(z,a|θ) Q ) represents the current Critic network's value estimate for any state z and action a, with parameter θ. Q ; This represents the gradient of the Q-value with respect to action a, measuring the impact of the action change on the value; a = μ(z) means replacing a with the action output by the Actor network; θ represents the gradient of the Actor network output with respect to its own parameters. μ Represents the parameters of the Actor network;

[0107] Soft update of the target network (Polyak averaging) allows the target network parameters to slowly follow the changes in the main network parameters, improving training stability. The formula is as follows:

[0108] θ Q′ ←τθ Q +(1-τθ Q′ ;

[0109] θ μ′ ←τθ μ +(1-τ)θ μ′ ;

[0110] Where τ∈(0,1) represents the soft update rate, which takes the value 0.005 and is used to smooth parameter updates and prevent oscillations.

[0111] The autonomous inspection and return control method and system for sewage pipeline robots of the present invention have the following advantages:

[0112] (i) This invention uses multimodal sensors to collect pipeline environmental data and uses a dynamic principal component analysis module to achieve real-time dimensionality reduction of high-dimensional perception data, thereby extracting key environmental features, improving the system's decision-making efficiency and response speed. The liquid neural network has superior time-series modeling capabilities and can effectively adapt to the dynamic changes in the environmental state in the sewage pipeline, improving the robot's path planning and obstacle avoidance capabilities in complex environments. Furthermore, the intelligent control strategy designed in conjunction with the deep deterministic policy gradient (DDPG) algorithm achieves highly robust and adaptive path decision-making.

[0113] (II) This invention introduces a trajectory tracing module and an autonomous return mechanism based on DDPG, which solves the problem that the robot cannot return safely under extreme conditions such as insufficient power or communication interruption. It significantly improves the practicality and safety of the system and is applicable to scenarios such as underground sewage pipe networks and industrial wastewater channels. It has broad application prospects and important engineering value in improving inspection efficiency and intelligence level.

[0114] (III) The intelligent robot system of this invention integrates efficient perception, intelligent decision-making, autonomous return and trajectory tracing capabilities. It can achieve stable, reliable and continuous siltation detection tasks in complex sewage pipeline environments and has the ability to save itself in abnormal conditions. In response to the problems of complex underground sewage pipeline environment, difficult passage and limited communication, the system integrates multiple intelligent algorithms to realize the robot's integrated perception and decision-making and full-process autonomous control in restricted environments.

[0115] (iv) This invention proposes an autonomous sewage pipeline exploration robot system that integrates Dynamic Principal Component Analysis (PCA), Liquid Neural Network (LNN), and Deep Reinforcement Learning (DRL). It combines a differential motion model to achieve intelligent path control. To solve the return-to-home problem during exploration, this invention designs a trajectory tracing module and integrates Graph Attention Network (GAT) and reinforcement learning methods to formulate an efficient return-to-home strategy. The system structure is modular and highly intelligent, significantly reducing the complexity of manual operation and improving the success rate and autonomy of sewage pipeline blockage exploration tasks. It is suitable for complex and dynamic underground pipe network environments.

[0116] (V) This invention uses a liquid neural network to model time-series state data. Its neuron state is dynamically updated through continuous time differentiation. It has the ability to memorize based on state history and can efficiently learn control strategies in dynamic environments, thereby improving the stability and generalization ability of the robot in complex pipeline environments. Attached Figure Description

[0117] The invention will be further described below with reference to the accompanying drawings.

[0118] Appendix Figure 1 A schematic diagram of the autonomous inspection and return control system for sewage pipeline robots;

[0119] Appendix Figure 2 A flowchart for the cruise state;

[0120] Appendix Figure 3 A flowchart for trajectory tracing;

[0121] Appendix Figure 4 A flowchart for the return-to-home status;

[0122] Appendix Figure 5 This is a schematic diagram of the structure of Liquid Neural Network (LNN) and Deep Reinforcement Learning (DRL) models. Detailed Implementation

[0123] The following detailed description of the autonomous inspection and return control method and system for sewage pipeline robots of the present invention is based on the accompanying drawings and specific embodiments.

[0124] Example 1:

[0125] This embodiment provides a method for autonomous inspection and return control of a sewage pipeline robot, as detailed below:

[0126] S1. Data Dimensionality Reduction: Receive high-dimensional environmental and motion state data from multimodal sensors, and use Dynamic Principal Component Analysis (Dynamic PCA) to perform real-time dimensionality reduction on the high-dimensional environmental and motion state data. Extract key features of the robot's own state information and the internal state information of the pipeline to construct a low-dimensional state vector.

[0127] S2. Deep Reinforcement Learning Control Based on Liquid Neural Network: The liquid neural network (LNN) is used to model the dynamic environmental state changes, and the deep deterministic policy gradient (DDPG) algorithm is combined to realize path planning and control. The state space is defined as the action space, and a multi-factor reward function is designed to guide the robot to complete the task.

[0128] S3, Trajectory Tracing: Record the robot's historical movement trajectory in the pipeline, combine spatial change threshold and trajectory interpolation algorithm to compress the trajectory, retain key path feature points, and identify and store key event nodes such as pollution sources and frequent obstacles to ensure that high-precision path reconstruction and event backtracking can still be achieved under limited memory resources.

[0129] S4. Return-to-home strategy based on graph attention network and reinforcement learning (GAT+DRL): When the robot needs to return to home, a trajectory graph model is constructed based on the robot's historical trajectory. Key path nodes and trajectory compression points are defined as nodes in the graph, and the trajectory order and feasible paths are used as edges of the graph. The node embedding is updated by combining graph attention network (GAT) to enhance the perception of key areas. The return-to-home path is planned based on reinforcement learning algorithm to improve return-to-home efficiency and safety.

[0130] The specific data dimensionality reduction in step S1 of this embodiment is as follows:

[0131] S101. Collect the robot's own state information and the internal state information of the pipeline; wherein, the robot's own state information includes the current linear velocity, current angular velocity, pose information, three-axis acceleration, three-axis angular acceleration, task completion status and previous historical actions; the internal state information of the pipeline includes the distance to obstacles in each direction, local slope, pipeline curvature and passage probability; the robot's own motion state is shown in Table 1 and the internal state of the pipeline is shown in Table 2.

[0132] Table 1 Self-Motion Status Table

[0133]

[0134] Table 2. Internal Status of Pipelines

[0135]

[0136] S102. Construct the original state vector s t Specifically: s t = [Robot's own state, pipe's internal environment state];

[0137] S103. The original state vector s is obtained through dynamic principal component analysis. t Perform dimensionality reduction to obtain the low-dimensional state vector x t The formula is: in, This represents the mean of the original state vector; This represents the matrix used to project the original state vector into a k-dimensional space through the covariance matrix.

[0138] The deep reinforcement learning control based on liquid neural networks in step S2 of this embodiment is as follows:

[0139] S201. Define the state space: Define the state space as a low-dimensional state vector x. t ;

[0140] S202. Define the motion space: Define the range of linear and angular velocities output by the current motion as the motion space, denoted as a. t =[v t ,ω t Where v represents linear velocity, v∈[-2,2], in meters per second; ω represents angular velocity, ω∈[-1,1], in rad / second;

[0141] S203. Design the reward function: This function integrates multiple factors to guide the robot to move efficiently forward, avoid obstacles, and complete the exploration task. The formula is as follows:

[0142] R t =W1×r forward +W2×r safe +W3×r task +W4×r smooth ;

[0143] Where, r forward The reward value represents the incentive to move in the target direction; r safe This represents the reward value for maintaining a safe distance and avoiding obstacles; r task This represents the task completion rate of the pluggable sensing module, i.e., the target task completion rate; r smoothThe reward value represents the oscillation penalty; a larger reward value indicates more drastic changes, i.e., severe "jitter," requiring penalty. W1, W2, W3, and W4 are the weights of the reward value for encouraging movement in the target direction, the reward value for maintaining a safe distance, the reward value for obstacle avoidance, the weight of the task completion rate of the pluggable sensing module, and the weight of the oscillation penalty, respectively. Normalization processing is performed before calculating the reward value for encouraging movement in the target direction, the reward value for maintaining a safe distance, the reward value for obstacle avoidance, the task completion rate of the pluggable sensing module, and the oscillation penalty, as detailed below:

[0144] r forward =V t ×cos(θ t );

[0145] Among them, V t θ represents the robot's current speed. t Cos(θ) represents the angle between the robot's current direction of motion and the target direction. t The ) indicates whether the movement is toward the target. The closer to 1, the better. If the movement is toward the target, the reward is positive; if it is in the opposite direction, the reward is negative.

[0146]

[0147] Where, d t Indicates the distance between the robot and the nearest obstacle; d safe This indicates the set minimum safe distance threshold; if you get too close to an obstacle, you will receive a negative reward.

[0148]

[0149] Where, n t n represents the number of tasks actually completed at each time step. total This indicates the number of tasks expected to be completed this time;

[0150] r smooth =(a t -a t-1 ) 2 ;

[0151] Among them, a t This indicates the action at the current time step; a t-1 It indicates the action at the previous time step.

[0152] Compared with traditional RNNs and LSTMs, the liquid neural network in step S2 of this embodiment has the following key advantages: Continuous temporal modeling: It describes neuron dynamics through differential equations, which can more naturally simulate state evolution in continuous time; Memory ability for state changes: Neuron states have dynamic memory, which is suitable for handling the dependence of state sequences in navigation tasks; High data efficiency: It can still learn effective strategies in small sample environments; Strong stability: It avoids problems such as unstable learning and gradient explosion / vanishing in traditional RNNs.

[0153] As attached Figure 2 and 5 As shown, the liquid neural network in this embodiment includes a state encoder, an Actor network, a Critic network, an Actor target network, and a Critic target network;

[0154] A Liquid Neural Network (LNN) is used to process complex temporal inputs and model the impact of historical states on the current decision. The dynamic update formula for the LNN is as follows:

[0155]

[0156] z t =u t ;

[0157] Where, x t μ represents a low-dimensional state vector. t Represents the internal state of a neuron; Δt represents the time elapsed since the walk; τ(x) represents the time elapsed since the walk. t ) represents the state-dependent time constant, allowing the network to respond to different inputs at different speeds; W in W represents the input weight matrix; rec The weight matrix represents the recursive connections (hidden layers); b represents the bias vector; f(.) represents the non-linear activation function; and the final output z... t This is the encoded state, used for subsequent policy evaluation;

[0158] Actor networks are used to output actions based on state encoders, as shown in the following formula:

[0159] a t =μ(z) t |θ μ );

[0160] Where μ represents the policy function; z t This represents the temporal features extracted by the state encoder; a t This indicates the currently selected action, i.e., the values ​​of linear velocity and angular velocity; θ μ Represents the learnable parameters of the Actor network;

[0161] The Critic network, used as an action-value function, evaluates the quality of the current state-action pair, denoted as: Q(z) t ,a t |θ Q );

[0162] Where Q(·) represents the action-value function; z t This represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q The parameters of the Critic network are used to update the Actors and serve as an estimate of the objective function's return.

[0163] The Actor and Critic target networks are used to construct stable target values. Specifically, a slowly updated Actor and Critic target network is introduced through a deep deterministic policy gradient algorithm. These networks correspond to copies of the Actor and Critic, respectively, and are defined as: μ′≈μ, Q′≈Q.

[0164] Where μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; and Q′ represents the Critic target network.

[0165] The training process of the liquid neural network in step S2 of this embodiment is as follows:

[0166] (1) The agent interacts with the environment: Each time the agent interacts, it selects an action based on the current state, and the environment provides feedback on the new state and reward. The "state-action-reward-new state" is recorded as data for subsequent learning.

[0167] (2) The TD objective is used to update the Critic network and represents "current reward + estimate of future reward". The formula for calculating the TD objective is as follows:

[0168] y t =r t +γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ );

[0169] Among them, y t This represents the TD target value for this step, used to train the Critic network; r t The z-axis represents the immediate reward obtained at the current time t; γ represents the discount factor, which takes a value between 0 and 1 and measures the importance of future rewards; Q′ represents the Critic target network, used to estimate the action value of the next state; zt+1 The state characteristics at the next time step are represented by μ′; μ′ represents the Actor target network, used to give the optimal action in the next state; θ μ′ θ represents the parameters of the Actor target network. Q′ These represent the parameters of the Critic target network;

[0170] (3) The loss to be minimized is: L Q =(Q(z) t ,a t )-y t ) 2 ;

[0171] Among them, L Q The loss function of the Critic network is Q(z), which measures the difference between the current Q value and the TD target. t ,a t ) represents the current state z of the Critic network. t and action a t Value estimation;

[0172] (4) The Actor network updates its parameters by maximizing the Q-value of the Critic network's output action, as shown in the following formula:

[0173]

[0174] in, The gradient of the objective function representing the Actor network parameters is used to update the Actor parameters; Q(z,a|θ) Q ) represents the current Critic network's value estimate for any state z and action a, with parameter θ. Q ; This represents the gradient of the Q-value with respect to action a, measuring the impact of the action change on the value; a = μ(z) means replacing a with the action output by the Actor network; θ represents the gradient of the Actor network output with respect to its own parameters. μ Represents the parameters of the Actor network;

[0175] (5) Soft update the target network (Polyak average). The parameters of the target network slowly follow the changes in the parameters of the main network, improving training stability. The formula is as follows:

[0176] θ Q′ ←τθ Q +(1-τθ Q′ ;

[0177] θ μ′ ←τθ μ +(1-τ)θ μ′ ;

[0178] Where τ∈(0,1) represents the soft update rate, which takes the value 0.005 and is used to smooth parameter updates and prevent oscillations.

[0179] As attached Figure 3 As shown, the trajectory tracing in step S3 of this embodiment is as follows:

[0180] S301. Real-time trajectory recording and compressed storage: Let the robot's position at time t be X. t Its trajectory sequence is represented as: τ={X1,X2,…,X n};

[0181] S302. To reduce memory burden, trajectory compression is performed using linear interpolation combined with a keypoint-preserving algorithm. The formula is as follows: Where δ represents the spatial variation threshold, the compression function keeps the path accuracy error within a preset value;

[0182] S303, Key Node Marking and Storage: The robot identifies pollution source point P through sensors. 污染源 ={p 污染源i}、Barrier source hotspot P 障碍源 ={p 障碍源i The key events and key node set are defined as: k = P 污染源 ∪P 障碍源 The key node set will be stored first and will not be affected by trajectory compression, for reference in the return planning and review after the patrol mission is completed;

[0183] S304. Trajectory Reconstruction Capability: This capability utilizes compressed trajectories and key node sets to reconstruct approximate trajectories, achieving near-realistic path backtracking. This is represented as:

[0184] As attached Figure 4 As shown, the return-to-home strategy based on graph attention network and reinforcement learning in step S4 of this embodiment is as follows:

[0185] S401. Environment Graph Construction: Construct a pipeline graph model G = (V, E); where the node set V contains trajectory compression points. All positions and key node k; using The construction method ensures that key nodes are perceived first, improving the interpretability and safety of the return path; the edge set E consists of trajectory order or feasible paths, and is assigned obstacle risk and distance weights.

[0186] S402, Graph Attention Embedding Extraction: LeakyReLU (Leaky Rectified Linear Unit) is a commonly used neural network activation function, expressed as: Where α is a constant with a value of 0.01; a graph attention mechanism is used to update the node embeddings, and each node embedding h... i The transformation formula is as follows:

[0187]

[0188] Where, α ij Let represent the attention weight of node j towards node i; W represents the linear transformation matrix of the graph attention layer, used to project the original features into a new feature space; h i Let represent the original feature vector of node i; 'a' represents the parameter vector of the attention mechanism, used to score the correlation between node pairs; N(i) represents the set of neighboring nodes of node i; the node feature update formula is: h i ′=σ(∑ j∈N(i) α ij Wh j ); where h i ′ represents the updated node features, incorporating neighbor node information; σ represents the activation function; W represents the learnable weight matrix used for feature transformation; h j This represents the current feature vector of neighbor node j;

[0189] S403, Reinforcement Learning Modeling, as detailed below:

[0190] S40301, State-space design: State vector s t From the current position node h i The equation is composed of ′ and environmental state, and the formula is as follows:

[0191]

[0192] Among them, h i ' represents the embedded representation of the current node after GAT encoding; θ t Indicates the current orientation angle; e t This represents the current remaining energy (scalar). This represents the graph distance from the current location to the return target point; rask t Indicates the risk value of the current location or path;

[0193] S40302, Motion Space Design: The range of linear and angular velocities output by the current motion is defined as the motion space, denoted as a. t =[v t ,ω t Where v represents linear velocity, v∈[-2,2], in meters per second; ω represents angular velocity, ω∈[-1,1], in rad / second;

[0194] S40303, Reward function design, the formula is as follows:

[0195]

[0196] Where; λ i Indicates the reward weight; This represents the graph distance from the current location to the return target point; rask t Indicates the risk value of the current location or path; collision t Indicates whether a collision has occurred; isdone indicates whether the return-to-base destination has been reached;

[0197] S40304, Policy Network Design: including state encoder, Actor network, Critic network, Actor target network and Critic target network;

[0198] The state encoder (LNN, Liquid Neural Network) is used to handle complex temporal inputs and model the impact of historical states on the current decision. The dynamic update formula for the state encoder is as follows:

[0199]

[0200] z t =u t ;

[0201] Where, x t μ represents a low-dimensional state vector. t Represents the internal state of a neuron; Δt represents the time elapsed since the walk; τ(x) represents the time elapsed since the walk. t ) represents the state-dependent time constant, allowing the network to respond to different inputs at different speeds; W in W represents the input weight matrix; rec The weight matrix represents the recursive connections (hidden layers); b represents the bias vector; f(.) represents the non-linear activation function; and the final output z... t This is the encoded state, used for subsequent policy evaluation;

[0202] Actor networks are used to output actions based on state encoders, as shown in the following formula:

[0203] a t =μ(z) t |θ μ );

[0204] Where μ represents the policy function; z t This represents the temporal features extracted by the state encoder; a t This indicates the currently selected action, i.e., the values ​​of linear velocity and angular velocity; θμ Represents the learnable parameters of the Actor network;

[0205] The Critic network, used as an action-value function, evaluates the quality of the current state-action pair, denoted as: Q(z) t ,a t |θ Q );

[0206] Where Q(·) represents the action-value function; z t This represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q The parameters of the Critic network are used to update the Actors and serve as an estimate of the objective function's return.

[0207] The Actor and Critic target networks are used to construct stable target values. Specifically, a slowly updated Actor and Critic target network is introduced through a deep deterministic policy gradient algorithm. These networks correspond to copies of the Actor and Critic, respectively, and are defined as: μ′≈μ, Q′≈Q.

[0208] Where μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; and Q′ represents the Critic target network.

[0209] The training process of the liquid neural network in this embodiment is as follows:

[0210] (1) The agent interacts with the environment: Each time the agent interacts, it selects an action based on the current state, and the environment provides feedback on the new state and reward. The "state-action-reward-new state" is recorded as data for subsequent learning.

[0211] (2) The TD objective is used to update the Critic network and represents "current reward + estimate of future reward". The formula for calculating the TD objective is as follows:

[0212] y t =r t +γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ );

[0213] Among them, y t This represents the TD target value for this step, used to train the Critic network; r tThe z-axis represents the immediate reward obtained at the current time t; γ represents the discount factor, which takes a value between 0 and 1 and measures the importance of future rewards; Q′ represents the Critic target network, used to estimate the action value of the next state; z t+1 The state characteristics at the next time step are represented by μ′; μ′ represents the Actor target network, used to give the optimal action in the next state; θ μ′ θ represents the parameters of the Actor target network. Q′ These represent the parameters of the Critic target network;

[0214] (3) The loss to be minimized is: L Q =(Q(z) t ,a t )-y t ) 2 ;

[0215] Among them, L Q The loss function of the Critic network is Q(z), which measures the difference between the current Q value and the TD target. t ,a t ) represents the current state z of the Critic network. t and action a t Value estimation;

[0216] (4) The Actor network updates its parameters by maximizing the Q-value of the Critic network's output action, as shown in the following formula:

[0217]

[0218] in, The gradient of the objective function representing the Actor network parameters is used to update the Actor parameters; Q(z,a|θ) Q ) represents the current Critic network's value estimate for any state z and action a, with parameter θ. 4 ; This represents the gradient of the Q-value with respect to action a, measuring the impact of the action change on the value; a = μ(z) means replacing a with the action output by the Actor network; θ represents the gradient of the Actor network output with respect to its own parameters. μ Represents the parameters of the Actor network;

[0219] (5) Soft update the target network (Polyak average). The parameters of the target network slowly follow the changes in the parameters of the main network, improving training stability. The formula is as follows:

[0220] θ Q′ ←τθ Q +(1-τθ Q′ ;

[0221] θ μ′ ←τθ μ +(1-τ)θ μ′ ;

[0222] Where τ∈(0,1) represents the soft update rate, which takes the value 0.005 and is used to smooth parameter updates and prevent oscillations.

[0223] The pluggable sensing module in this embodiment has the following functions:

[0224] ① It supports various heterogeneous sensor modules and can provide task rewards to the main control module to improve collaborative perception and decision-making capabilities; among them, heterogeneous sensor modules include image acquisition sensors, scanning area sensors, obstacle detection sensors and pollution recognition sensors;

[0225] ② Supports heterogeneous task modules, which can be flexibly mounted and unmounted and decoupled from the main control system (vehicle control); among which, the heterogeneous task modules include pollution detection module, crack identification module and temperature monitoring module;

[0226] ③ Unified interface specifications, supporting module registration / deregistration;

[0227] ④ Output task results or events (such as "contamination detected");

[0228] ⑤ Feedback reward signal (transmitted to the DRL module).

[0229] Example 2:

[0230] As attached Figure 1 As shown, this embodiment provides an autonomous inspection and return control system for a sewage pipeline robot. This system is used to implement the autonomous inspection and return control method for the sewage pipeline robot as described in Embodiment 1. The system includes:

[0231] The data dimensionality reduction module is used to receive high-dimensional environmental and motion state data from multimodal sensors, and uses the Dynamic Principal Component Analysis (Dynamic PCA) method to perform real-time dimensionality reduction on the high-dimensional environmental and motion state data, extract key features of the robot's own state information and the internal state information of the pipeline, and construct a low-dimensional state vector.

[0232] The deep reinforcement learning control module based on liquid neural network is used to model dynamic environmental state changes using liquid neural network (LNN), combine deep deterministic policy gradient (DDPG) algorithm to realize path planning and control, define the state space as action space, and design multi-factor reward function to guide the robot to complete the task.

[0233] The trajectory tracing module is used to record the robot's historical movement trajectory in the pipeline. It combines spatial change threshold and trajectory interpolation algorithm to compress the trajectory, retain key path feature points, and identify and store key event nodes such as pollution sources and frequent obstacles, so as to ensure that high-precision path reconstruction and event backtracking can still be achieved under the condition of limited memory resources.

[0234] The Return-to-Home Strategy (GAT+DRL) module, based on Graph Attention Network and Reinforcement Learning, is used to construct a trajectory graph model based on the robot's historical motion trajectory when the robot needs to return to home. Key path nodes and trajectory compression points are defined as nodes in the graph, and trajectory order and feasible paths are defined as edges of the graph. The node embedding is updated by combining Graph Attention Network (GAT) to enhance the perception of key areas, and the return-to-home path is planned based on reinforcement learning algorithm to improve return-to-home efficiency and safety.

[0235] As attached Figure 5 As shown, the liquid neural network in this embodiment includes a state encoder, an Actor network, a Critic network, an Actor target network, and a Critic target network;

[0236] A Liquid Neural Network (LNN) is used to process complex temporal inputs and model the impact of historical states on the current decision. The dynamic update formula for the LNN is as follows:

[0237]

[0238] z t =u t ;

[0239] Where, x t μ represents a low-dimensional state vector. t Represents the internal state of a neuron; Δt represents the time elapsed since the walk; τ(x) represents the time elapsed since the walk. t ) represents the state-dependent time constant, allowing the network to respond to different inputs at different speeds; W in W represents the input weight matrix; rec The weight matrix represents the recursive connections (hidden layers); b represents the bias vector; f(.) represents the non-linear activation function; and the final output z... t This is the encoded state, used for subsequent policy evaluation;

[0240] Actor networks are used to output actions based on state encoders, as shown in the following formula:

[0241] a t =μ(z) t |θ μ );

[0242] Where μ represents the policy function; z t This represents the temporal features extracted by the state encoder; a t This indicates the currently selected action, i.e., the values ​​of linear velocity and angular velocity; θ μ Represents the learnable parameters of the Actor network;

[0243] The Critic network, used as an action-value function, evaluates the quality of the current state-action pair, denoted as: Q(z) t ,a t |θ Q );

[0244] Where Q(·) represents the action-value function; z t This represents the temporal features extracted by the state encoder; a t Indicates the currently selected action; θ Q The parameters of the Critic network are used to update the Actors and serve as an estimate of the objective function's return.

[0245] The Actor and Critic target networks are used to construct stable target values. Specifically, a slowly updated Actor and Critic target network is introduced through a deep deterministic policy gradient algorithm. These networks correspond to copies of the Actor and Critic, respectively, and are defined as: μ′≈μ, Q′≈Q.

[0246] Where μ represents the Actor network; μ′ represents the Actor target network; Q represents the Critic network; and Q′ represents the Critic target network.

[0247] The training process of the liquid neural network in this embodiment is as follows:

[0248] The agent interacts with the environment: each time the agent interacts, it selects an action based on the current state, and the environment provides feedback on the new state and reward. The "state-action-reward-new state" is recorded as data for subsequent learning.

[0249] The TD objective is used to update the Critic network and represents "current reward + estimate of future reward". The formula for calculating the TD objective is as follows:

[0250] y t =r t +γQ′(z t+1 ,μ′(z t+1 |θ μ′ )|θ Q′ );

[0251] Among them, y tThis represents the TD target value for this step, used to train the Critic network; r t The z-axis represents the immediate reward obtained at the current time t; γ represents the discount factor, which takes a value between 0 and 1 and measures the importance of future rewards; Q′ represents the Critic target network, used to estimate the action value of the next state; z t+1 The state characteristics at the next time step are represented by μ′; μ′ represents the Actor target network, used to give the optimal action in the next state; θ μ′ θ represents the parameters of the Actor target network. Q′ These represent the parameters of the Critic target network;

[0252] The minimum loss is: L Q =(Q(z) t ,a t )-y t ) 2 ;

[0253] Among them, L Q The loss function of the Critic network is Q(z), which measures the difference between the current Q value and the TD target. t ,a t ) represents the current state z of the Critic network. t and action a t Value estimation;

[0254] The Actor network updates its parameters by maximizing the Q-value of the Critic network's output action, as shown in the following formula:

[0255]

[0256] in, The gradient of the objective function representing the Actor network parameters is used to update the Actor parameters; Q(z,a|θ) Q ) represents the current Critic network's value estimate for any state z and action a, with parameter θ. Q ; This represents the gradient of the Q-value with respect to action a, measuring the impact of the action change on the value; a = μ(z) means replacing a with the action output by the Actor network; θ represents the gradient of the Actor network output with respect to its own parameters. μ Represents the parameters of the Actor network;

[0257] Soft update of the target network (Polyak averaging) allows the target network parameters to slowly follow the changes in the main network parameters, improving training stability. The formula is as follows:

[0258] θ Q′ ←τθ Q+(1-τθ Q′ ;

[0259] θ μ′ ←τθ μ +(1-τ)θ μ′ ;

[0260] Where τ∈(0,1) represents the soft update rate, which takes the value 0.005 and is used to smooth parameter updates and prevent oscillations.

[0261] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A sewer pipeline robot autonomous inspection and return control method, characterized in that, The method is as follows: Data dimensionality reduction: Receive high-dimensional environmental and motion state data from multimodal sensors, and use dynamic principal component analysis to perform real-time dimensionality reduction on the high-dimensional environmental and motion state data. Extract key features of the robot's own state information and the internal state information of the pipeline to construct a low-dimensional state vector. Deep reinforcement learning control based on liquid neural network: The liquid neural network is used to model the dynamic environmental state changes, and the path planning and control are realized by combining the deep deterministic policy gradient algorithm. The state space is defined as the action space, and a multi-factor reward function is designed to guide the robot to complete the task. Trajectory tracing: Record the robot's historical motion trajectory in the pipeline, combine spatial change threshold and trajectory interpolation algorithm to compress the trajectory, retain key path feature points, and identify and store key event nodes to ensure that high-precision path reconstruction and event backtracking can still be achieved under limited memory resources; A return-to-home strategy based on graph attention network and reinforcement learning: When the robot needs to return to home, a trajectory graph model is constructed based on the robot's historical trajectory. Key path nodes and trajectory compression points are defined as nodes in the graph, and the trajectory order and feasible paths are used as edges of the graph. The node embedding is updated by combining graph attention network to enhance the perception of key areas, and the return-to-home path is planned based on reinforcement learning algorithm. The liquid neural network includes a state encoder, an Actor network, a Critic network, an Actor target network, and a Critic target network. State encoders are used to handle complex time-series inputs and model the impact of state history on the current decision. The dynamic update formula for the state encoder is as follows: ； ； in, Represents a low-dimensional state vector; Indicates the internal state of a neuron; This indicates that the time elapsed since the walk; This represents a state-dependent time constant, allowing the network to respond to different inputs at different speeds; This represents the input weight matrix; This represents the weight matrix between recursive connections; Represents the bias vector; This represents the non-linear activation function, and the final output. This is the encoded state, used for subsequent policy evaluation; Actor networks are used to output actions based on state encoders, as shown in the following formula: ； in, Represents the policy function; This represents the temporal features extracted by the state encoder; This indicates the currently selected action, namely the values ​​of linear velocity and angular velocity; Represents the learnable parameters of the Actor network; The Critic network, used as an action-value function, evaluates the quality of the current state-action pair, and is represented as: ; in, Represents the action value function; This represents the temporal features extracted by the state encoder; Indicates the currently selected action; The parameters of the Critic network are used to update the Actor and serve as an estimate of the objective function's return. The Actor and Critic target networks are used to construct stable target values. Specifically, they are introduced using a deep deterministic policy gradient algorithm, resulting in slowly updated Actor and Critic target networks, which correspond to copies of Actor and Critic, respectively, and are defined as follows: ; in, Represents an Actor network; Represents the Actor target network; Represents the Critic network; Represents the Critic target network; The training process of a liquid neural network is as follows: The agent interacts with the environment: In each interaction, the agent selects an action based on the current state, and the environment provides feedback on the new state and reward. The "state-action-reward-new state" is recorded as data for subsequent learning. The TD objective is used to update the Critic network and represents "current reward + estimate of future reward". The formula for calculating the TD objective is as follows: ； in, This represents the TD target value for this step, used to train the Critic network; This represents the immediate reward obtained at the current time t; This represents the discount factor, with a value between 0 and 1, which measures the importance of future rewards; This represents the Critic target network, used to estimate the action value of the next state; Indicates the state characteristics at the next moment; This represents the target network of the Actor, used to determine the optimal action in the next state; Represents the parameters of the Actor's target network; These represent the parameters of the Critic target network; The loss to be minimized is: ; in, This represents the loss function of the Critic network, which measures the difference between the current Q value and the TD target. This indicates the current state of the Critic network. and actions Value estimation; The Actor network updates its parameters by maximizing the Q-value of the Critic network's output action, as shown in the following formula: ； in, This represents the gradient of the objective function for updating the Actor network parameters; This represents the current Critic network's value estimate for any state z and action a, with parameters as follows: ; This represents the gradient of the Q-value with respect to action a, measuring the impact of changes in action on value; This means replacing 'a' with the action output by the Actor network; This represents the gradient of the Actor network output with respect to its own parameters; Represents the parameters of the Actor network; Soft updates to the target network, where the target network parameters slowly follow the changes in the main network parameters, improve training stability. The formula is as follows: ； ； Where τ∈(0,1) represents the soft update rate, which is used to smooth parameter updates and prevent oscillations.

2. The method for autonomous inspection and return control of sewage pipeline robots according to claim 1, characterized in that, The specific details of data dimensionality reduction are as follows: Collect robot's own state information and the state information inside the pipeline; the robot's own state information includes current linear velocity, current angular velocity, pose information, three-axis acceleration, three-axis angular acceleration, task completion status, and previous historical actions; the state information inside the pipeline includes obstacle distances in each direction, local slope, pipeline curvature, and probability of passage. Construct the original state vector Specifically: ; The original state vector is obtained through dynamic principal component analysis. Perform dimensionality reduction to obtain a low-dimensional state vector. The formula is: ;in, This represents the mean of the original state vector; This represents the matrix used to project the original state vector into a k-dimensional space through the covariance matrix.

3. The method for autonomous inspection and return control of sewage pipeline robots according to claim 1, characterized in that, The specific details of deep reinforcement learning control based on liquid neural networks are as follows: Define the state space: Define the state space as a low-dimensional state vector. ; Define the motion space: Define the range of linear and angular velocities output by the current motion as the motion space, denoted as: ;in, Indicates linear velocity. The unit is meters per second; Indicates angular velocity. The unit is rad / second; Design a reward function: This function integrates multiple factors to guide the robot to move efficiently forward, avoid obstacles, and complete its exploration task. The formula is as follows: ； in, The reward value represents the incentive to move in the target direction; This represents the reward value for maintaining a safe distance and avoiding obstacles; This indicates the task completion rate of the pluggable sensing module, i.e., the target task completion rate. This indicates an oscillation penalty; the larger the reward value, the more drastic the change, i.e., severe "jitter," which requires penalty. , , and These are the weights for the reward value that encourages movement in the target direction, the weights for maintaining a safe distance, the weights for obstacle avoidance, the weights for the task completion rate of the pluggable sensing module, and the weights for oscillation penalty. Normalization processing is performed before calculating the reward value for encouraging movement in the target direction, maintaining a safe distance, the reward value for obstacle avoidance, the task completion rate of the pluggable sensing module, and the oscillation penalty, as detailed below: = ； in, Indicates the robot's current speed; This represents the angle between the robot's current direction of motion and the target direction; This indicates whether the movement is toward the target; the closer to 1, the better. If the movement is toward the target, the reward is positive; otherwise, it is negative. ； in, Indicates the distance between the robot and the nearest obstacle; This indicates the set minimum safe distance threshold; if you get too close to an obstacle, you will receive a negative reward. ； in, Indicates the number of tasks actually completed at each time step; This indicates the number of tasks expected to be completed this time; ; in, This indicates the action at the current time step; It indicates the action at the previous time step.

4. The method for autonomous inspection and return control of sewage pipeline robots according to claim 1, characterized in that, The trajectory tracing is as follows: Real-time trajectory recording and compressed storage: Let the robot's position at time t be... Its trajectory sequence is represented as: ; To reduce memory load, linear interpolation combined with a keypoint-preserving algorithm is used for trajectory compression, as shown in the formula: ;in, This represents the spatial variation threshold, ensuring that the compression function maintains path accuracy error within a preset value. Key node marking and storage: Robots identify pollution sources using sensors. Obstacles and Hotspots The key events, the set of key nodes are defined as: The key node set will be stored first and will not be affected by trajectory compression, for reference in return planning and review after the patrol mission is completed; Trajectory reconstruction capability: By utilizing compressed trajectories and key node sets to reconstruct approximate trajectories, an approximate path backtracking capability is achieved, represented as: .

5. The method for autonomous inspection and return control of a sewage pipeline robot according to claim 1, characterized in that, The return-to-home strategy based on graph attention networks and reinforcement learning is as follows: Environment Graph Construction: Building a Pipeline Graph Model The node set V contains trajectory compression points. All locations and key nodes ;use The construction method ensures that key nodes are perceived first, improving the interpretability and safety of the return path; the edge set E consists of trajectory order or feasible paths, and is assigned obstacle risk and distance weights. Graph attention embedding extraction: A graph attention mechanism is used to update node embeddings, with each node embedding... The transformation formula is as follows: ； in, Represents a node For nodes The attention weights; W represents the linear transformation matrix of the graph attention layer, used to project the original features into a new feature space. Represents a node The original feature vector; The parameter vector represents the attention mechanism and is used to score the correlation between node pairs. Represents a node The set of neighboring nodes; the node feature update formula is: ;in, This represents the updated node characteristics, incorporating information from neighboring nodes; Indicates the activation function; This represents a learnable weight matrix used for feature transformation; Representing neighboring nodes The current feature vector; Reinforcement learning modeling, specifically as follows: State-space design: State vector From the current position node It is composed of environmental conditions, and the formula is as follows: ； in, This represents the embedded representation of the current node after GAT encoding; Indicates the current orientation angle; Indicates the current remaining energy; This represents the graph distance from the current location to the return destination. Indicates the risk value of the current location or path; Motion space design: The range of linear and angular velocities of the current motion output is defined as the motion space, expressed as... ;in, Indicates linear velocity. The unit is meters per second; Indicates angular velocity. The unit is rad / second; The reward function is designed as follows: + ； in; Indicates the reward weight; This represents the graph distance from the current location to the return destination. Indicates the risk value of the current location or path; Indicates whether a collision has occurred; Indicate whether the return destination has been reached; Policy network design includes state encoder, Actor network, Critic network, Actor target network, and Critic target network.

6. The method for autonomous inspection and return control of a sewage pipeline robot according to claim 3, characterized in that, The pluggable sensing module has the following functions: ① It supports various heterogeneous sensor modules and can provide task rewards to the main control module to improve collaborative perception and decision-making capabilities; among them, heterogeneous sensor modules include image acquisition sensors, scanning area sensors, obstacle detection sensors and pollution recognition sensors; ② Supports heterogeneous task modules, which can be flexibly mounted and unmounted and decoupled from the main control system; among them, the heterogeneous task modules include a pollution detection module, a crack identification module, and a temperature monitoring module; ③ Unified interface specifications, supporting module registration / deregistration; ④ Output task results or events; ⑤ Provide feedback and reward signals.

7. A control system for autonomous inspection and return of a sewage pipeline robot, characterized in that, This system is used to implement the autonomous inspection and return control method for sewage pipeline robots as described in any one of claims 1 to 6; the system includes: The data dimensionality reduction module is used to receive high-dimensional environmental and motion state data from multimodal sensors, and to perform real-time dimensionality reduction on the high-dimensional environmental and motion state data using dynamic principal component analysis. It extracts key features of the robot's own state information and the internal state information of the pipeline, and constructs a low-dimensional state vector. The deep reinforcement learning control module based on liquid neural network is used to model dynamic environmental state changes using liquid neural network, combine deep deterministic policy gradient algorithm to realize path planning and control, define state space as action space, and design multi-factor reward function to guide robot to complete task. The trajectory tracing module is used to record the robot's historical motion trajectory in the pipeline. It combines spatial change threshold and trajectory interpolation algorithm to compress the trajectory, retain key path feature points, and identify and store key event nodes to ensure that high-precision path reconstruction and event backtracking can still be achieved under limited memory resources. The return-to-home strategy module based on graph attention network and reinforcement learning is used to construct a trajectory graph model based on the robot's historical trajectory when the robot needs to return to home. Key path nodes and trajectory compression points are defined as nodes in the graph, and trajectory order and feasible paths are defined as edges of the graph. The node embedding is updated by combining graph attention network to enhance the perception of key areas, and the return-to-home path is planned based on reinforcement learning algorithm. The liquid neural network includes a state encoder, an Actor network, a Critic network, an Actor target network, and a Critic target network. State encoders are used to handle complex time-series inputs and model the impact of state history on the current decision. The dynamic update formula for the state encoder is as follows: ； ； in, Represents a low-dimensional state vector; Indicates the internal state of a neuron; This indicates that the time elapsed since the walk; This represents a state-dependent time constant, allowing the network to respond to different inputs at different speeds; This represents the input weight matrix; This represents the weight matrix between recursive connections; Represents the bias vector; This represents the non-linear activation function, and the final output. This is the encoded state, used for subsequent policy evaluation; Actor networks are used to output actions based on state encoders, as shown in the following formula: ； in, Represents the policy function; This represents the temporal features extracted by the state encoder; This indicates the currently selected action, namely the values ​​of linear velocity and angular velocity; Represents the learnable parameters of the Actor network; The Critic network, used as an action-value function, evaluates the quality of the current state-action pair, and is represented as: ; in, Represents the action value function; This represents the temporal features extracted by the state encoder; Indicates the currently selected action; The parameters of the Critic network are used to update the Actor and serve as an estimate of the objective function's return. The Actor and Critic target networks are used to construct stable target values. Specifically, they are introduced using a deep deterministic policy gradient algorithm, resulting in slowly updated Actor and Critic target networks, which correspond to copies of Actor and Critic, respectively, and are defined as follows: ; in, Represents an Actor network; Represents the Actor target network; Represents the Critic network; Represents the Critic target network; The training process of a liquid neural network is as follows: The agent interacts with the environment: In each interaction, the agent selects an action based on the current state, and the environment provides feedback on the new state and reward. The "state-action-reward-new state" is recorded as data for subsequent learning. The TD objective is used to update the Critic network and represents "current reward + estimate of future reward". The formula for calculating the TD objective is as follows: ； in, This represents the TD target value for this step, used to train the Critic network; This represents the immediate reward obtained at the current time t; This represents the discount factor, with a value between 0 and 1, which measures the importance of future rewards; This represents the Critic target network, used to estimate the action value of the next state; Indicates the state characteristics at the next moment; This represents the target network of the Actor, used to determine the optimal action in the next state; Represents the parameters of the Actor's target network; These represent the parameters of the Critic target network; The loss to be minimized is: ; in, This represents the loss function of the Critic network, which measures the difference between the current Q value and the TD target. This indicates the current state of the Critic network. and actions Value estimation; The Actor network updates its parameters by maximizing the Q-value of the Critic network's output action, as shown in the following formula: ； in, This represents the gradient of the objective function for updating the Actor network parameters; This represents the current Critic network's value estimate for any state z and action a, with parameters as follows: ; This represents the gradient of the Q-value with respect to action a, measuring the impact of changes in action on value; This means replacing 'a' with the action output by the Actor network; This represents the gradient of the Actor network output with respect to its own parameters; Represents the parameters of the Actor network; Soft updates to the target network, where the target network parameters slowly follow the changes in the main network parameters, improve training stability. The formula is as follows: ； ； Where τ∈(0,1) represents the soft update rate, which is used to smooth parameter updates and prevent oscillations.

Images