Deep reinforcement learning control system for pipeline leak detection robot path planning and obstacle avoidance

Abstract

The application relates to the technical field of pipeline robot control, and particularly discloses a deep reinforcement learning control system for path planning and obstacle avoidance of a pipeline leak detection robot, which synchronously collects pipeline inner wall images through a depth camera and a polarization camera, and records a leakage concentration time sequence signal; performs refraction disturbance decoupling mapping on the images to obtain a refraction invariant feature map; performs space-time registration on the feature map and inertia, wheel speed and concentration data, respectively extracts an obstacle avoidance potential field and a concentration attractive potential field, and fuses the two fields into a composite embedded tensor; solves an action probability density distribution based on the tensor, embeds a refraction disturbance penalty term to suppress a false obstacle response, finally outputs a driving instruction, and updates a target point according to a leakage concentration gradient direction; the application can suppress water surface refraction virtual image interference, and realizes collaborative control of obstacle avoidance and leak detection.

Description

Technical Field

[0001] This invention relates to the field of pipeline robot control technology, specifically to a deep reinforcement learning control system for path planning and obstacle avoidance of a pipeline leak detection robot. Background Technology

[0002] In the field of pipeline leak detection robots, industrial vision technology is widely used to acquire environmental information about the inner wall of pipelines to support the robot's path planning and obstacle avoidance control. Existing deep reinforcement learning methods typically rely on simulation environments for policy training, using domain randomization to improve the model's adaptability to different pipeline operating conditions.

[0003] In wet pipes containing gas-liquid two-phase flow, water surface refraction simultaneously causes visual interference from false obstacles and conflicts with the task objectives at the decision level. On the one hand, the robot may misjudge the refracted virtual image as a real obstacle and thus perform ineffective avoidance. On the other hand, traditional separate control methods may unconsciously deviate from the direction of the leakage concentration gradient when avoiding real obstacles, resulting in mutual hindrance between obstacle avoidance behavior and leak detection.

[0004] This invention suppresses the virtual image response by decoupling the refraction perturbation mapping and dynamically updates the target point by weighted fusion of the obstacle avoidance potential field and the concentration attraction potential field, thereby simultaneously solving the two coupled problems of refraction misjudgment and task coordination under the same control framework. Summary of the Invention

[0005] The purpose of this invention is to provide a deep reinforcement learning control system for path planning and obstacle avoidance of a pipeline leak detection robot, so as to solve the problems mentioned above.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A deep reinforcement learning control system for path planning and obstacle avoidance of a pipeline leak detection robot, including:

[0008] The data acquisition module acquires raw depth image data streams and polarization image data streams of the inner wall of the pipeline, and simultaneously records the leakage concentration time-series signal output by the gas sensor;

[0009] The refraction decoupling transformation module performs refraction perturbation decoupling mapping on the original depth image data stream and polarization image data stream to obtain a refraction-invariant feature map that is insensitive to water surface refraction.

[0010] The composite embedding generation module performs spatiotemporal registration and gradient field calculation on the refraction invariant feature map, inertial measurement data, wheel speed feedback data, and leakage concentration time series signal to generate a composite embedding tensor including obstacle avoidance potential field and concentration attraction potential field.

[0011] The action distribution solution module, based on the composite embedding tensor, solves the action probability density distribution that minimizes the obstacle avoidance potential field and maximizes the concentration attraction potential field through the policy value iteration field, and embeds a refraction perturbation penalty term into the distribution to suppress false obstacle responses.

[0012] The collaborative control output module outputs drive wheel speed commands and steering angle commands based on probability density distribution, and updates the target point of the path planning based on the direction of leakage concentration gradient, thereby realizing collaborative control of obstacle avoidance and leak detection.

[0013] As a further aspect of the present invention: obtaining a refraction-invariant feature map that is insensitive to water surface refraction specifically includes:

[0014] The original depth image data stream and polarization image data stream are respectively subjected to discrete wavelet decomposition to extract their respective high-frequency detail components and low-frequency approximation components.

[0015] Phase correlation matching is performed on the two high-frequency detail components to calculate the local phase shift field caused by the refraction disturbance.

[0016] Based on the local phase shift field, inverse wavelet transform compensation is performed on the low-frequency approximate components of the original depth image data stream to reconstruct a refraction-invariant feature map that eliminates refraction distortion.

[0017] As a further aspect of the present invention: the calculation of the local phase shift field generated by the refraction disturbance specifically includes:

[0018] The two high-frequency detail components are transformed to the polar complex domain, and their respective amplitude and phase spectra are extracted.

[0019] Perform a sliding cross-correlation based on a ring window on the two phase spectra to search for the peak phase difference in the neighborhood of each pixel;

[0020] Median filtering is applied to the peak phase difference to smooth the edges, and a local phase shift field is output.

[0021] As a further aspect of the present invention: the generation process of the composite embedding tensor is as follows:

[0022] The refraction invariance feature map, inertial measurement data, wheel speed feedback data, and leakage concentration time series signal are aligned by bilinear interpolation according to their respective timestamps and unified to the same time reference.

[0023] Under a unified benchmark, the distance gradient field after anisotropic diffusion is calculated on the refractive invariant feature map as the obstacle avoidance potential field, and the local concentration gradient field is calculated on the leakage concentration time series signal as the concentration attraction potential field.

[0024] The obstacle avoidance potential field and the concentration attraction potential field are fused at the pixel level to generate a composite embedding tensor.

[0025] As a further aspect of the present invention: the calculation of the distance gradient field after anisotropic diffusion on the refractive invariant feature map as the obstacle avoidance potential field, and the calculation of the local concentration gradient field on the leakage concentration time series signal as the concentration attraction potential field, specifically includes:

[0026] Apply a diffusion coefficient guided by the structure tensor to the refraction-invariant feature map to perform directional smoothing along the pipe axis and circumferential direction, suppressing texture noise while preserving obstacle edges;

[0027] A signed distance transformation is performed on the smoothed feature map, and the distance gradient field is extracted along the gradient descent direction as the obstacle avoidance potential field.

[0028] The leakage concentration time series signal is subjected to differential operation with an adaptive time window. After removing the concentration pulsation noise, the spatial first-order partial derivative is calculated, and the local concentration gradient field is output.

[0029] As a further aspect of the present invention: the specific embedding process of the refraction perturbation penalty term is as follows:

[0030] Calculate the pixel-wise absolute residual between the original depth image data stream and the refraction-invariant feature map to obtain the refraction perturbation residual map;

[0031] Block max pooling is performed on the refraction perturbation residual map to generate a binary mask to label candidate regions of false obstacles;

[0032] In the action probability density distribution, a log probability reduction is applied to the action candidate values ​​that fall within the binary mask region to complete the embedding of the refraction perturbation penalty term.

[0033] As a further aspect of the present invention: the generation of binary mask markers for false obstacle candidate regions specifically includes:

[0034] The refraction perturbation residual map is divided into overlapping image blocks, and the maximum residual value is calculated within each image block.

[0035] The maximum residual values ​​of all image patches are used to construct a saliency map, and adaptive double threshold segmentation is used to extract high-response regions.

[0036] Morphological closing operations are performed on high-response regions to fill holes, and a binary mask is output to mark false obstacle candidate regions.

[0037] As a further aspect of the present invention: the step of updating the target point of the path planning based on the direction of the leakage concentration gradient specifically includes:

[0038] The leakage concentration gradient direction is decomposed into a first component along the pipe axis and a second component perpendicular to the pipe wall circumferentially.

[0039] Based on the angle difference between the first component and the current direction of travel, calculate the displacement increment of the target point along the direction of travel of the pipeline;

[0040] Based on the magnitude of the second component, a circumferential offset is accumulated on the radial offset perpendicular to the current path, and the updated target point coordinates are output.

[0041] As a further aspect of the present invention: the decomposition specifically includes:

[0042] Based on the roll and pitch angles in the inertial measurement data and the displacement increments in the wheel speed feedback data, a local cylindrical coordinate system for the pipeline is established with the robot's current position as the origin. The axial direction of the local cylindrical coordinate system is tangential to the pipeline centerline, and the circumferential direction is along the inner circumference of the pipeline.

[0043] The leakage concentration gradient direction vector is projected onto the axial and circumferential directions of the local cylindrical coordinate system, and the axial scalar components and circumferential scalar components are extracted respectively.

[0044] A sliding median filter is applied to the axial scalar component, and an amplitude dead zone is set for the circumferential scalar component to eliminate minor jitter. The first and second components are then output.

[0045] The beneficial effects of this invention are:

[0046] (1) A refraction-invariant feature map is generated by decoupling the refraction perturbation and a binary mask is constructed based on the residual between the original depth image and the refraction-invariant feature map to suppress false obstacle responses. This enables the robot to effectively distinguish between the virtual image generated by water surface refraction and the real obstacle when performing obstacle avoidance in a pipe containing condensate or gas-liquid two-phase flow, thereby reducing ineffective avoidance actions caused by light deflection and maintaining a stable travel path.

[0047] (2) The obstacle avoidance potential field and the concentration attraction potential field are weighted and fused, and the target point of the path planning is dynamically updated according to the direction of the leakage concentration gradient. This enables the robot to actively move towards the direction of increasing leakage concentration while avoiding obstacles such as protrusions and deposits on the inner wall of the pipeline. This achieves the coordination of obstacle avoidance behavior and leakage source search behavior, and shortens the time required for the robot to locate the leakage point from entering the pipeline. Attached Figure Description

[0048] The invention will now be further described with reference to the accompanying drawings.

[0049] Figure 1 This is a system block diagram of the present invention;

[0050] Figure 2 This is a flowchart of the local phase shift field in this invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Please see Figure 1 As shown, this invention is a deep reinforcement learning control system for path planning and obstacle avoidance of a pipeline leak detection robot, comprising:

[0053] The data acquisition module acquires raw depth image data streams and polarization image data streams of the inner wall of the pipeline, and simultaneously records the leakage concentration time-series signal output by the gas sensor;

[0054] The refraction decoupling transformation module performs refraction perturbation decoupling mapping on the original depth image data stream and polarization image data stream to obtain a refraction-invariant feature map that is insensitive to water surface refraction.

[0055] The composite embedding generation module performs spatiotemporal registration and gradient field calculation on the refraction invariant feature map, inertial measurement data, wheel speed feedback data, and leakage concentration time series signal to generate a composite embedding tensor including obstacle avoidance potential field and concentration attraction potential field.

[0056] The action distribution solution module, based on the composite embedding tensor, solves the action probability density distribution that minimizes the obstacle avoidance potential field and maximizes the concentration attraction potential field through the policy value iteration field, and embeds a refraction perturbation penalty term into the distribution to suppress false obstacle responses.

[0057] The collaborative control output module outputs drive wheel speed commands and steering angle commands based on probability density distribution, and updates the target point of the path planning based on the direction of leakage concentration gradient, thereby realizing collaborative control of obstacle avoidance and leak detection.

[0058] The data acquisition module acquires raw depth image data streams and polarization image data streams of the inner wall of the pipe, and simultaneously records the leakage concentration time-series signal output by the gas sensor, specifically including:

[0059] During the movement of the pipeline leak detection robot, a depth camera and a polarization camera, fixedly mounted at the robot's front end, synchronously acquire visual data of the pipeline's inner wall. The depth camera utilizes active infrared structured light, outputting a raw depth image data stream with a resolution of 640×480 pixels at a frame rate of 30 frames per second. Each pixel value represents the distance from the corresponding spatial point to the camera. The polarization camera is arranged parallel to the depth camera, with a rotatable linear polarization filter at its front end. The rotation angle alternates between 0 degrees and 90 degrees, outputting a polarization image data stream at the same frame rate. Each pixel value records the light intensity information after polarization modulation. Both cameras are connected to the same clock source via a hardware trigger line, ensuring that the timestamp deviation of each frame is less than 1 millisecond.

[0060] Simultaneously, a semiconductor gas sensor is installed in the middle of the robot body, with the sensor's air inlet facing the pipe axis and equipped with a dust filter. The gas sensor continuously measures the concentration of leaked gas in the pipe at a sampling frequency of 10 Hz and generates a leak concentration time-series signal. Each data point is accompanied by a time stamp provided by the robot controller's crystal oscillator. The aforementioned raw depth image data stream, polarization image data stream, and leak concentration time-series signal are all transmitted to the storage unit through the robot's internal data bus, and are stored aligned according to a unified time reference, providing synchronous input data for subsequent refraction perturbation decoupling mapping and gradient field calculation.

[0061] Please see Figure 2 As shown, in the refraction decoupling transformation module, refraction perturbation decoupling mapping is performed on the original depth image data stream and the polarization image data stream to obtain a refraction-invariant feature map that is insensitive to water surface refraction. Specifically, this includes:

[0062] The specific process of performing refraction perturbation decoupling mapping on the original depth image data stream and the polarization image data stream is as follows:

[0063] Each frame of the original depth image data stream and polarization image data stream is subjected to discrete wavelet decomposition. The discrete wavelet decomposition uses the Haar wavelet basis. After a first-order decomposition of the image, four sub-bands are obtained: low-frequency approximation components, horizontal high-frequency detail components, vertical high-frequency detail components, and diagonal high-frequency detail components. The high-frequency detail components extracted in this step refer to the sum of the high-frequency components in the horizontal, vertical, and diagonal directions, while the low-frequency approximation components refer to the low-frequency sub-bands that retain the main energy information of the original image.

[0064] The two high-frequency detail components obtained from the decomposition are input into the phase correlation matching stage to calculate the local phase shift field caused by refraction perturbation. This local phase shift field describes the pixel displacement distribution of the polarized image data stream relative to the original depth image data stream under the effect of water surface refraction.

[0065] Based on the calculated local phase shift field, a compensatory inverse wavelet transform is performed on the low-frequency approximation components of the original depth image data stream. Specifically, the compensation method is as follows: using the local phase shift field as a displacement map, each pixel in the low-frequency approximation component is resampled according to the direction and distance indicated by the shift field. Then, the compensated low-frequency approximation components and the high-frequency detail components of the original depth image data stream are reconstructed using inverse wavelets, thereby obtaining a refraction-invariant feature map that eliminates refraction distortion.

[0066] In the above process, the calculation of the local phase shift field generated by the refraction disturbance is achieved through the following specific steps:

[0067] The high-frequency detail components of the original depth image data stream and the high-frequency detail components of the polarization image data stream are transformed to the polar complex domain, respectively. During the transformation, the gray value of each pixel is regarded as the amplitude in the complex domain, and the angle value determined by the row and column coordinates of the pixel in the image is used as the initial phase, thereby constructing a two-dimensional matrix in complex form. The amplitude spectrum and phase spectrum of this complex matrix are extracted, where the amplitude spectrum reflects the intensity of local texture in the image, and the phase spectrum reflects the spatial location information of local structure in the image.

[0068] A sliding cross-correlation operation based on a ring window is performed on the two phase spectra. Specifically, for each target pixel, multiple concentric ring regions with gradually increasing radii are selected as the search neighborhood. Within each ring region, the cross-correlation value between the phase spectrum of the original depth image data stream and the phase spectrum of the polarization image data stream is calculated. The position offset where the cross-correlation value reaches its maximum within the ring region is searched; this offset is the peak phase difference at that pixel. By traversing all pixels, an initial phase difference peak map is obtained.

[0069] Median filtering and edge-preserving smoothing are applied to the peak phase difference image. Median filtering uses a 5-pixel × 5-pixel sliding window, sorting all phase difference values ​​within the window and replacing the center pixel value with the median to eliminate isolated noise points. Edge-preserving smoothing employs a bilateral filtering method, with a spatial domain standard deviation of 2 pixels and a grayscale domain standard deviation of 0.1 times the phase difference range, smoothing phase difference variations while preserving phase abrupt changes at obstacle edges. After these filtering and smoothing processes, the final local phase shift field is output. Each pixel value in this shift field represents the local displacement vector caused by water surface refraction at the corresponding location.

[0070] In the composite embedding generation module, the refraction-invariant feature map is spatiotemporally registered and gradient field calculated with inertial measurement data, wheel speed feedback data, and leakage concentration time-series signal to generate a composite embedding tensor including obstacle avoidance potential field and concentration attraction potential field, specifically including:

[0071] Spatiotemporal registration and bilinear interpolation alignment are performed: After the robot controller acquires the refraction-invariant feature map, the triaxial acceleration and triaxial angular velocity from the inertial measurement data, the left and right wheel speeds from the wheel speed feedback data, and the leakage concentration time series signal, it aligns them according to the timestamps of their respective data sources. Since different sensors have different sampling frequencies (30 Hz for depth and polarization cameras, 100 Hz for the inertial measurement unit, 50 Hz for the wheel speed encoder, and 10 Hz for the gas sensor), the 100 Hz timestamp of the inertial measurement data is used as a unified time reference. For each reference moment, the two nearest preceding and following refraction-invariant feature map data frames are found, and the interpolation feature map for that moment is calculated using bilinear interpolation. Linear interpolation is used for the wheel speed feedback data; and nearest neighbor interpolation is used for the leakage concentration time series signal to preserve concentration abrupt changes. After the above processing, all data are aligned temporally to the same 100 Hz reference, forming synchronized multi-source data frames.

[0072] Under a unified time reference, anisotropic diffusion-guided directional smoothing is performed on the refraction-invariant feature map. Specifically, for each frame of the refraction-invariant feature map, its structure tensor is first calculated. The structure tensor is a 2x2 matrix, where each element is obtained by weighting the spatial gradient field of the feature map using a Gaussian window. The window size is fixed at 5 pixels × 5 pixels, and the standard deviation of the Gaussian kernel is 1 pixel. A diffusion coefficient is calculated based on two eigenvalues ​​of the structure tensor. This diffusion coefficient controls the smoothing intensity along the axial and circumferential directions of the pipe. The expression for calculating the diffusion coefficient is: ;

[0073] It should be noted that K is a preset edge sensitivity constant. and These represent the two eigenvalues ​​of the structure tensor. The values ​​calculated from these eigenvalues ​​are... The value is used as the intensity factor of anisotropy and multiplied by the axial and circumferential diffusion rates, respectively. The axial diffusion rate is set to 0.8 and the circumferential diffusion rate is set to 0.2. When smoothing the direction of directional smoothing, smoothing is performed first along the pipe axis (i.e., the robot's forward direction) to suppress fine texture noise on the inner wall of the water pipe, while preserving the edge features of obstacles perpendicular to the axis.

[0074] A signed distance transformation is performed on the smoothed feature map, and the distance gradient field is extracted. The signed distance transformation refers to calculating the signed distance value from each non-edge pixel to the nearest edge, using obstacle edge pixels in the feature map as references. Distances inside the edge (obstacle region) are positive, and distances outside are negative. Euclidean distance is used during the transformation, and the distance value of each pixel is updated through two scans (forward and backward scans). After obtaining the signed distance map, the distance gradient field is extracted along the gradient descent direction. The gradient descent direction is the direction in which the distance value decreases the fastest, i.e., the direction from the obstacle edge outwards towards free space. Specifically, for each pixel, the distance difference between it and its eight neighboring pixels is calculated. The direction with the largest and negative difference is taken as the gradient direction, and the unit vector of this gradient direction is assigned to that pixel to form the obstacle avoidance potential field. The expression for calculating the magnitude of the gradient field is: ;

[0075] in, Indicates the signed distance value. Represents the partial derivative in the horizontal direction. This represents the partial derivative in the vertical direction. The partial derivative is calculated using the central difference method: for internal pixels, the horizontal partial derivative equals the difference between the right and left pixel distances, divided by 2; the vertical partial derivative equals the difference between the lower and upper pixel distances, divided by 2. For boundary pixels, one-sided difference is used. The final result... The values ​​constitute the gradient magnitude of each pixel in the obstacle avoidance potential field, and the direction is determined by the signs of the two partial derivatives.

[0076] A local concentration gradient field is calculated from the leakage concentration time-series signal. An adaptive time window differential operation is used, with the window length dynamically adjusted according to the concentration change rate: first, the variance of the concentration values ​​over the past 10 sampling points (corresponding to 1 second) is calculated. If the variance exceeds a preset threshold of 0.05, the window is shortened to 5 sampling points; otherwise, 10 sampling points are maintained. Within the current window, the concentration values ​​are differentially analyzed to obtain the concentration change rate. Then, based on the displacement change calculated from the robot's wheel speed feedback data, the time-domain change rate is converted into a spatial gradient. Specifically, the concentration change rate at each sampling moment is divided by the instantaneous velocity at that moment (calculated from the average rotational speed of the left and right wheels) to obtain the spatial gradient of the concentration along the pipe axis. Considering the circumferential oscillation of the robot and combining the roll angle change in the inertial measurement data, a circumferential correction is performed on the concentration gradient, ultimately outputting a two-dimensional local concentration gradient field, where each point contains both axial and circumferential gradient components.

[0077] The obstacle avoidance potential field and the concentration attraction potential field are fused at the pixel level to generate a composite embedding tensor. The weighting coefficients of the weighted fusion adopt a dynamic balancing strategy: initially, the obstacle avoidance potential field weight is 0.7, and the concentration attraction potential field weight is 0.3; when the leakage concentration time-series signal exceeds the background noise threshold of 0.01 for three consecutive frames, the weight of the concentration attraction potential field is gradually increased to 0.6, and the weight of the obstacle avoidance potential field is correspondingly decreased to 0.4. During fusion, for each pixel position, the gradient vector of the obstacle avoidance potential field is multiplied by its corresponding weight and then added to the gradient vector of the concentration attraction potential field, resulting in the composite gradient vector for that pixel. All composite gradient vectors of pixels are organized spatially according to the original feature map, forming a three-dimensional tensor (height × width × 2), i.e., the composite embedding tensor, where the two channels of the third dimension represent the fused axial and circumferential motion guidance information, respectively. This composite embedding tensor will serve as the input data for subsequent action probability density distribution calculation.

[0078] In the action distribution solution module, based on the composite embedding tensor, the action probability density distribution that minimizes the obstacle avoidance potential field and maximizes the concentration attraction potential field is solved through the policy value iteration field. A refraction perturbation penalty term is embedded in the distribution to suppress false obstacle responses. Specifically, this includes:

[0079] The embedding process for constructing the refraction perturbation penalty term involves acquiring the original depth image data stream and the refraction-invariant feature map, then calculating the absolute difference between them pixel by pixel. Specifically, for pixels at the same coordinate position in each frame, the pixel value in the original depth image data stream is subtracted from the corresponding pixel value in the refraction-invariant feature map, and the absolute value of the difference is taken to obtain the refraction perturbation residual value for that pixel. After performing the above operation on all pixels, a refraction perturbation residual map of the same size as the original image is formed. Each value in this residual map reflects the degree of distortion caused by water surface refraction at that location; the more significant the distortion, the larger the residual value.

[0080] To generate a binary mask, the refractive perturbation residual map is divided into multiple overlapping image blocks, each 8 pixels × 8 pixels in size, with a 4-pixel overlap between blocks, meaning adjacent blocks overlap by 4 pixels horizontally and vertically. Within each block, the refractive perturbation residual values ​​of all pixels are iterated, and the maximum value is used as the representative residual value for that block. The representative residual values ​​of all blocks are then reorganized according to the spatial order of the original blocks to form a saliency map smaller than the original image. The value at each position in the saliency map corresponds to the maximum residual value of a given image block.

[0081] An adaptive dual-threshold segmentation technique is used to extract high-response regions from the saliency map: First, the average value of all values ​​in the entire saliency map is calculated, denoted as the first threshold; then, the average value of all values ​​greater than the first threshold is calculated, denoted as the second threshold. Pixels in the saliency map with values ​​greater than the second threshold are marked as 1 (representing high-response regions), and the remaining pixels are marked as 0, thus obtaining a binary high-response region map. Subsequently, morphological closing operations are performed on this binary high-response region map to fill the holes.

[0082] The closing operation consists of two steps: First, a dilation operation is performed on the binary image, using a circular kernel with a radius of 3 pixels as the structuring element. This expands the high-response region outward by one pixel, connecting adjacent isolated regions and filling small holes. Then, an erosion operation is performed, again using a circular kernel with a radius of 3 pixels, shrinking the dilated region inward back to approximately its original size while preserving the connectivity of the filled areas. After the closing operation, a binary mask with the same size as the original saliency map is output. Regions with a value of 1 in this mask are marked as false obstacle candidate regions.

[0083] A refraction perturbation penalty term is embedded into the action probability density distribution. The action probability density distribution is obtained by solving the policy value iterative field, which gives the probability value of the robot performing each possible action in the current state. For each spatial region marked as 1 in the binary mask, a logarithmic probability reduction is applied to all action candidate values ​​within that region. Specifically, the natural logarithm of the probability value of each action falling into the false obstacle candidate region is taken, and then the resulting logarithm is multiplied by a penalty coefficient (fixed at 0.5). Finally, the reduced logarithm is converted back into a probability value. For regions marked as 0 in the binary mask, the action probability value remains unchanged. After this processing, the probability of actions belonging to the false obstacle candidate region being executed is significantly reduced, thereby suppressing the robot's false obstacle response and preventing it from making invalid avoidance actions due to virtual images generated by water surface refraction. The action probability density distribution after the penalty term embedding will serve as the basis for the final output drive command.

[0084] It should be noted that the "policy value iteration field" refers to a deep reinforcement learning network that adopts an Actor-Critic architecture. The Actor network outputs the action probability density distribution, and the Critic network outputs the state value function. The two networks update the policy parameters iteratively to simultaneously minimize the obstacle avoidance potential field and maximize the concentration attraction potential field.

[0085] In the collaborative control output module, drive wheel speed and steering angle commands are output based on probability density distribution, and the target point of the path planning is updated based on the direction of the leakage concentration gradient, realizing collaborative control of obstacle avoidance and leak detection, specifically including:

[0086] Based on the motion probability density distribution, the controller outputs drive wheel speed commands and steering angle commands. From the solved motion probability density distribution, the controller selects the motion with the highest probability value as the action to be executed at the current moment. This motion contains two dimensions: linear velocity and angular velocity components. The linear velocity component is converted into the average rotational speed of the left and right drive wheels through the robot's kinematic model. The specific conversion relationship is: divide the linear velocity by the drive wheel radius, multiply by 60, divide by pi, and then divide by twice pi to obtain the rotational speed per minute, which is the reference rotational speed shared by both wheels. The angular velocity component is converted into a steering angle command. The conversion method is: multiply the angular velocity by the control cycle length (the control cycle is fixed at 0.02 seconds) to obtain the angle value that the robot should rotate within that cycle. A positive value indicates a left turn, and a negative value indicates a right turn. The reference rotational speed and steering angle commands are sent to the drivers of the left and right drive motors respectively to realize robot motion control.

[0087] The target point of the path planning is updated based on the direction of the leakage concentration gradient. After each control cycle, the coordinates of the pre-set target point of the path planning are updated according to the current leakage concentration gradient direction, enabling the robot to approach the leakage source. The update process includes four steps: establishing a local cylindrical coordinate system, component decomposition, displacement increment calculation, and circumferential offset accumulation.

[0088] A local cylindrical coordinate system is established within the pipe with the robot's current position as the origin. The controller reads the roll and pitch angles from the inertial measurement data, as well as the displacement increments from the wheel speed feedback data. The roll angle represents the robot's tilt angle around its forward axis, the pitch angle represents the robot's pitch angle around its lateral axis, and the displacement increment is the robot's movement distance obtained by multiplying the average left and right wheel speeds by the control cycle duration. Based on the above data, three axes of the local cylindrical coordinate system are defined: the axial direction is the tangential direction along the pipe's centerline, with the positive direction of the robot's current forward direction taken as the positive axial direction; the circumferential direction is the circumferential direction along the inner wall of the pipe, with the robot's top as the circumferential zero-degree reference and the clockwise direction as the positive circumferential direction; the radial direction is perpendicular to the inner wall of the pipe and points towards the center. The origin of this local cylindrical coordinate system is fixed at the robot's current geometric center.

[0089] The leakage concentration gradient direction vector is projected onto the axial and circumferential axes of the local cylindrical coordinate system. The leakage concentration gradient direction vector is obtained by preprocessing the three-dimensional concentration change rate measured by the gas sensor array, and is represented as a three-dimensional vector in the robot's coordinate system. The controller places the starting point of this vector at the origin of the local cylindrical coordinate system and calculates the projected lengths of the vector in the axial and circumferential directions. The axial projected length is equal to the dot product of the vector and the axial unit vector, which is the axial scalar component; a positive value indicates the gradient points in the robot's forward direction, and a negative value indicates it points backward. The circumferential projected length is equal to the dot product of the vector and the circumferential unit vector, which is the circumferential scalar component; a positive value indicates the gradient points in the clockwise direction, and a negative value indicates it points in the counterclockwise direction.

[0090] A sliding median filter is applied to the axial scalar component. The sliding median filter uses a window of length 5, storing the axial scalar component values ​​calculated over the last 5 control cycles. When a new value enters, the oldest value in the window is discarded. Then, the 5 values ​​in the window are sorted by size, and the value at the median is taken as the current output value. An amplitude dead zone is set for the circumferential scalar component, with an absolute value of 0.02. This means that when the absolute value of the circumferential scalar component is less than 0.02, it is forcibly set to zero to eliminate jitter caused by sensor noise or small concentration fluctuations. After the above processing, the first component (the filtered axial scalar component) and the second component (the dead-zone-processed circumferential scalar component) are output.

[0091] Based on the angle difference between the first component and the current direction of travel, the displacement increment of the target point along the pipeline's forward direction is calculated. The angle difference is calculated as follows: the sign of the first component determines the angle difference; a positive value indicates the target point should move forward, and a negative value indicates it should move backward. The absolute value of the first component is multiplied by a scaling factor (fixed at 0.5) to obtain the radian value of the angle difference. Then, the tangent of the angle difference is multiplied by the robot's displacement increment to obtain the displacement increment of the target point along the pipeline's forward direction, in meters. If the angle difference is positive, the axial coordinate of the target point is increased by this displacement increment; if it is negative, it is decreased.

[0092] Based on the amplitude of the second component, a circumferential offset is accumulated on the radial offset perpendicular to the current path. The radial offset represents the distance of the target point from the pipe centerline, with an initial value of zero. The amplitude (absolute value) of the second component is multiplied by a circumferential scaling factor (which is fixed at 0.3) to obtain the circumferential offset in radians. This radian value is converted to an arc length (multiplied by the pipe radius, which is the nominal radius of the actual pipe being inspected, for example, 0.2 meters), which is the accumulated change in the radial offset. The sign of the second component determines the offset direction: a positive sign indicates accumulation in a clockwise direction, and a negative sign indicates accumulation in a counterclockwise direction. The updated radial offset is combined with the axial displacement increment to output the updated target point coordinates, represented as axial coordinates and circumferential angle values ​​in a local cylindrical coordinate system. These target point coordinates are fed back to the path planner as the position the robot expects to reach in the next control cycle, thus achieving coordinated control of obstacle avoidance and leak detection.

[0093] The working principle of this invention is as follows: During the movement of the pipeline leak detection robot, raw depth image data streams and polarization image data streams are simultaneously acquired by a depth camera and a polarization camera, while a gas sensor records the leakage concentration time-series signal. A refraction perturbation decoupling mapping based on discrete wavelet decomposition and phase correlation matching is performed on the two image data streams to obtain a refraction-invariant feature map that eliminates water surface refraction distortion. This refraction-invariant feature map is then spatiotemporally registered with inertial measurement data, wheel speed feedback data, and the leakage concentration time-series signal. Anisotropic diffusion guided by a structure tensor and a signed distance transform are applied to the refraction-invariant feature map to extract the obstacle avoidance potential field. An adaptive time window difference is used on the leakage concentration time-series signal to extract the concentration attraction potential field. Finally, the two data streams are combined... Weighted fusion is performed to obtain a composite embedding tensor. Based on the composite embedding tensor, the action probability density distribution that minimizes the obstacle avoidance potential field and maximizes the concentration attraction potential field is solved through the policy value iteration field. Block max pooling and adaptive double threshold segmentation are performed based on the residual map between the original depth image and the refraction invariant feature map to generate a binary mask that marks the false obstacle region. Log probability reduction is applied to this region in the distribution to suppress the refraction disturbance response. Finally, the drive wheel speed command and steering angle command are output based on the probability density distribution. At the same time, the leakage concentration gradient direction is decomposed into the axial component and the circumferential component of the pipeline. The forward displacement of the target point is updated according to the axial component and the radial offset is accumulated according to the circumferential component to achieve coordinated control of obstacle avoidance and leak detection.

[0094] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A deep reinforcement learning control system for pipeline leak detection robot path planning and obstacle avoidance, characterized in that, include: The data acquisition module acquires raw depth image data streams and polarization image data streams of the inner wall of the pipeline, and simultaneously records the leakage concentration time-series signal output by the gas sensor; The refraction decoupling transformation module performs refraction perturbation decoupling mapping on the original depth image data stream and polarization image data stream to obtain a refraction-invariant feature map that is insensitive to water surface refraction. The composite embedding generation module performs spatiotemporal registration and gradient field calculation on the refraction invariant feature map, inertial measurement data, wheel speed feedback data, and leakage concentration time series signal to generate a composite embedding tensor including obstacle avoidance potential field and concentration attraction potential field. The action distribution solution module, based on the composite embedding tensor, solves the action probability density distribution that minimizes the obstacle avoidance potential field and maximizes the concentration attraction potential field through the policy value iteration field, and embeds a refraction perturbation penalty term into the distribution to suppress false obstacle responses. The collaborative control output module outputs drive wheel speed and steering angle commands based on probability density distribution, and updates the target point of the path planning according to the direction of leakage concentration gradient, thereby realizing collaborative control of obstacle avoidance and leak detection.

2. The pipeline leak detection robot path planning and obstacle avoidance deep reinforcement learning control system according to claim 1, wherein, The process of obtaining a refraction-invariant feature map that is insensitive to water surface refraction specifically includes: The original depth image data stream and polarization image data stream are respectively subjected to discrete wavelet decomposition to extract their respective high-frequency detail components and low-frequency approximation components. Phase correlation matching is performed on the two high-frequency detail components to calculate the local phase shift field caused by the refraction disturbance. Based on the local phase shift field, inverse wavelet transform compensation is performed on the low-frequency approximate components of the original depth image data stream to reconstruct a refraction-invariant feature map that eliminates refraction distortion.

3. The deep reinforcement learning control system for path planning and obstacle avoidance of the pipeline leak detection robot according to claim 2, characterized in that, The calculation of the local phase shift field generated by the refraction disturbance specifically includes: The two high-frequency detail components are converted to the polar complex domain, and their respective amplitude and phase spectra are extracted. Perform a sliding cross-correlation based on a ring window on the two phase spectra to search for the peak phase difference in the neighborhood of each pixel; Median filtering is applied to the peak phase difference to smooth the edges, and a local phase shift field is output.

4. The deep reinforcement learning control system for path planning and obstacle avoidance of the pipeline leak detection robot according to claim 1, characterized in that, The process of generating the composite embedding tensor is as follows: The refraction invariance feature map, inertial measurement data, wheel speed feedback data, and leakage concentration time series signal are aligned by bilinear interpolation according to their respective timestamps and unified to the same time reference. Under a unified benchmark, the distance gradient field after anisotropic diffusion is calculated on the refractive invariant feature map as the obstacle avoidance potential field, and the local concentration gradient field is calculated on the leakage concentration time series signal as the concentration attraction potential field. The obstacle avoidance potential field and the concentration attraction potential field are fused at the pixel level to generate a composite embedding tensor.

5. The deep reinforcement learning control system for path planning and obstacle avoidance of the pipeline leak detection robot according to claim 4, characterized in that, The calculation of the distance gradient field after anisotropic diffusion from the refractive invariant feature map as the obstacle avoidance potential field, and the calculation of the local concentration gradient field from the leakage concentration time series signal as the concentration attraction potential field, specifically include: Apply a diffusion coefficient guided by the structure tensor to the refraction-invariant feature map to perform directional smoothing along the pipe axis and circumferential direction, suppressing texture noise while preserving obstacle edges; A signed distance transformation is performed on the smoothed feature map, and the distance gradient field is extracted along the gradient descent direction as the obstacle avoidance potential field. The leakage concentration time series signal is subjected to differential operation with an adaptive time window. After removing the concentration pulsation noise, the spatial first-order partial derivative is calculated, and the local concentration gradient field is output.

6. The deep reinforcement learning control system for path planning and obstacle avoidance of the pipeline leak detection robot according to claim 1, characterized in that, The specific embedding process for the refraction perturbation penalty term is as follows: Calculate the pixel-wise absolute residual between the original depth image data stream and the refraction-invariant feature map to obtain the refraction perturbation residual map; Block maximum pooling is performed on the refraction perturbation residual map to generate a binary mask to label false obstacle candidate regions; In the action probability density distribution, a log probability reduction is applied to the action candidate values ​​that fall within the binary mask region to complete the embedding of the refraction perturbation penalty term.

7. The deep reinforcement learning control system for path planning and obstacle avoidance of the pipeline leak detection robot according to claim 6, characterized in that, The generation of binary mask markers for false obstacle candidate regions specifically includes: The refraction perturbation residual map is divided into overlapping image blocks, and the maximum residual value is calculated within each image block. The maximum residual values ​​of all image patches are used to construct a saliency map, and adaptive double threshold segmentation is used to extract high-response regions. Morphological closing operations are performed on high-response regions to fill holes, and a binary mask is output to mark false obstacle candidate regions.

8. The deep reinforcement learning control system for path planning and obstacle avoidance of the pipeline leak detection robot according to claim 1, characterized in that, The target point for updating the path planning based on the direction of the leakage concentration gradient specifically includes: The leakage concentration gradient direction is decomposed into a first component along the pipe axis and a second component perpendicular to the pipe wall circumferentially. Based on the angle difference between the first component and the current direction of travel, calculate the displacement increment of the target point along the direction of travel of the pipeline; Based on the magnitude of the second component, a circumferential offset is accumulated on the radial offset perpendicular to the current path, and the updated target point coordinates are output.

9. The deep reinforcement learning control system for path planning and obstacle avoidance of the pipeline leak detection robot according to claim 8, characterized in that, The decomposition specifically includes: Based on the roll and pitch angles in the inertial measurement data and the displacement increments in the wheel speed feedback data, a local cylindrical coordinate system for the pipeline is established with the robot's current position as the origin. The axial direction of the local cylindrical coordinate system is tangential to the pipeline centerline, and the circumferential direction is along the inner circumference of the pipeline. The leakage concentration gradient direction vector is projected onto the axial and circumferential directions of the local cylindrical coordinate system, and the axial scalar components and circumferential scalar components are extracted respectively. A sliding median filter is applied to the axial scalar component, and an amplitude dead zone is set for the circumferential scalar component to eliminate minor jitter. The first and second components are then output.

Images