Pipeline internal wall inspection robot system and method

Abstract

The application provides a pipeline inner wall detection robot system and method, relates to the technical field of pipeline nondestructive testing, and comprises a pipeline inspection robot body, a lifting sliding table, a rotating holder, a binocular structured light vision module and a vehicle-mounted controller. The lifting sliding table is installed on the upper surface of the pipeline inspection robot body. The rotating holder is installed at the top end of the lifting sliding table, and the base of the rotating holder is fixedly connected with the moving platform of the lifting sliding table through bolts. The rotating holder comprises a stepping motor and a planetary reduction box and can continuously rotate around a vertical axis. The binocular structured light vision module comprises a binocular camera, a structured light projector and a substrate and is installed on the rotating holder. The vehicle-mounted controller is installed on the upper surface of the pipeline inspection robot body and is used for controlling the lifting sliding table to adjust the height. The vehicle-mounted controller is electrically connected with the lifting sliding table and the rotating holder respectively.

Description

Technical Field

[0001] This invention relates to the field of non-destructive testing technology for pipelines, and particularly to a robotic system and method for inspecting the inner wall of pipelines. Background Technology

[0002] With the development of intelligent manufacturing, robotics, and intelligent infrastructure operation and maintenance technologies, robots are increasingly being used in pipeline inspection and maintenance. Because urban pipe networks, petrochemical pipelines, and gas transmission pipelines are typically characterized by narrow spaces, complex environments, and difficulties for manual entry, using robots to enter the pipeline interior to complete inspection tasks can effectively improve inspection efficiency and reduce the risks to personnel. At the same time, as the safety requirements for urban infrastructure continue to increase, higher demands are being placed on the accuracy, efficiency, and automation level of pipeline internal wall defect detection.

[0003] Existing pipeline inspection technologies mainly include traditional non-destructive testing methods such as ultrasonic testing, eddy current testing, and magnetic flux leakage testing, as well as robotic inspection methods based on monocular vision, ordinary binocular vision, or deep learning image recognition. These methods can, to a certain extent, achieve image acquisition of the pipeline's inner wall, defect identification, or local defect detection. Some visual inspection methods can also be combined with robot motion to complete internal pipeline inspection operations.

[0004] However, existing technologies have core drawbacks: First, traditional non-destructive testing methods suffer from problems such as sensitivity to detection direction and slow detection speed; second, monocular vision or ordinary binocular vision methods are prone to problems such as difficulty in stereo matching and low accuracy of 3D reconstruction in environments with insufficient lighting and lack of texture on the inner wall of the pipe; in addition, existing deep learning detection models have a large computational load, making it difficult to deploy them in real time at the edge. If a large amount of image data is sent back to the host computer for processing, it is easily limited by the communication bandwidth inside the pipe, making it difficult to meet the real-time detection requirements. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method for detecting the inner wall of a pipeline, which can solve the problems of sensitivity to detection direction and slow detection speed in the prior art. Secondly, monocular vision or ordinary binocular vision methods are prone to problems such as difficulty in stereo matching and low accuracy of 3D reconstruction in environments with insufficient lighting and lack of texture on the inner wall of the pipeline. In addition, existing deep learning detection models have a large computational load and are difficult to deploy in real time at the edge. If a large amount of image data is sent back to the host computer for processing, it is easily limited by the communication bandwidth inside the pipeline, making it difficult to meet the technical problems of real-time detection requirements.

[0006] In a first aspect of the present invention, a pipeline inner wall inspection robot system is proposed, comprising: a pipeline inspection robot body 1, a lifting slide 2, a rotating gimbal 3, a binocular structured light vision module 4, and an on-board controller 5. The lifting slide 2 is installed on the upper surface of the pipeline inspection robot body 1; The rotating gimbal 3 is installed on the top of the lifting slide 2, and the base of the rotating gimbal 3 is fixedly connected to the moving platform of the lifting slide 2 by bolts. The rotating gimbal 3 includes a stepper motor and a planetary gearbox, and can rotate continuously around a vertical axis. The binocular structured light vision module 4 includes a binocular camera, a structured light projector, and a substrate, which are mounted on a rotating gimbal 3. The vehicle-mounted controller 5 is installed on the upper surface of the pipeline inspection robot body 1 and is used to control the height adjustment of the lifting slide 2; The vehicle-mounted controller 5 is electrically connected to the lifting slide 2 and the rotating gimbal 3 respectively.

[0007] A second aspect of this invention provides a method for inspecting the inner wall of a pipe using a robot, comprising: S1: Place the pipeline inspection robot body 1 in the pipeline to be inspected, and measure the distance from the inner wall of the pipeline to be inspected to the binocular structured light vision module 4 through the binocular structured light vision module 4. S2: Based on the distance, the diameter of the pipe to be inspected is estimated by the vehicle controller 5, and the inner wall radius of the pipe is determined based on the pipe diameter; S3: Based on the inner wall radius of the pipe, control the lifting slide 2 to move the binocular structured light vision module 4 in the vertical direction, so that the binocular structured light vision module 4 is raised to the axial height of the pipe to be inspected; S4: The on-board controller 5 controls the pipeline inspection robot body 1 to move along the pipeline axis according to the preset step length, and the on-board controller 5 controls the rotating gimbal 3 to drive the binocular structured light vision module 4 to rotate to multiple preset angles in sequence. S5: At various preset angles, the vehicle-mounted controller 5 controls the structured light projector to project Gray code stripes and phase-shift stripes onto the inner wall of the pipe to be inspected, and controls the binocular camera to simultaneously acquire images of the inner wall of the pipe. S6: Input the image of the inner wall of the pipe into a lightweight semantic segmentation network, output a pixel-level defect mask, and extract the set of defect pixel coordinates in the pixel-level defect mask; S7: Calculate the wrapping phase based on the inner wall image using a four-step phase shift formula; S8: Determine whether the package phase is within the preset phase range. If so, the package phase is the absolute phase; otherwise, decode the inner wall image through the vehicle controller 5 to obtain the phase period order, and calculate the absolute phase based on the phase period order. S9: Based on the absolute phase and the inner radius of the pipe wall, limit the search range of the inner wall image of the pipe to determine the corresponding point pairs in the inner wall image of the pipe; S10: Based on the same point pairs, calculate the three-dimensional point cloud coordinates using the principle of binocular vision triangulation, and generate a three-dimensional point cloud model of the inner wall of the pipe based on the three-dimensional point cloud coordinates. S11: Input the set of defect pixel coordinates into the 3D point cloud model of the inner wall of the pipe, and output the defect area of ​​the pipe to be detected.

[0008] A third aspect of the present invention provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the pipe inner wall inspection robot method of the second aspect.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, a binocular structured light vision module is mounted on the pipeline inspection robot body, and a rotating gimbal is used to achieve multi-angle scanning detection. This allows the vision module to sequentially acquire images of the inner wall of the pipeline from different angles along the circumference, thereby improving the detection coverage and efficiency. Secondly, Gray code fringes and phase-shifted fringes are actively projected onto the inner wall of the pipeline using a structured light projector. The structured light encoded information assists in matching corresponding points in the left and right images, and a three-dimensional point cloud model of the inner wall of the pipeline is generated by combining the binocular vision triangulation principle, thereby improving the matching reliability and 3D reconstruction accuracy in weak texture environments. Furthermore, an embedded artificial intelligence processing platform with an integrated neural network processing unit is used in the onboard controller, and a lightweight semantic segmentation network is deployed. This allows the acquired pipeline inner wall images to be directly segmented and processed by the robot, reducing the large amount of original image backhaul, thereby improving detection real-time performance and reducing dependence on communication bandwidth. Attached Figure Description

[0010] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0011] Figure 1 This is a schematic diagram of a pipe inner wall inspection robot system provided in an embodiment of the present invention.

[0012] Figure 2 This is a schematic flowchart of a pipe inner wall inspection robot method provided in an embodiment of the present invention.

[0013] Explanation of reference numerals in the attached diagram: 1-Pipeline inspection robot body; 2-Lifting slide; 3-Rotating gimbal; 4-Binocular structured light vision module; 5-Vehicle controller. Detailed Implementation

[0014] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] The pipe inner wall inspection robot method provided by the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0016] Reference manual attached Figure 1 The diagram shows a structural schematic of a pipe inner wall inspection robot system provided in an embodiment of the present invention.

[0017] This invention provides a pipeline inner wall inspection robot system, including: a pipeline inspection robot body 1, a lifting slide 2, a rotating gimbal 3, a binocular structured light vision module 4, and a vehicle-mounted controller 5.

[0018] The lifting slide 2 is installed on the upper surface of the pipeline inspection robot body 1.

[0019] The rotating gimbal 3 is installed on the top of the lifting slide 2, and the base of the rotating gimbal 3 is fixedly connected to the moving platform of the lifting slide 2 by bolts.

[0020] The rotating gimbal 3 includes a stepper motor and a planetary gearbox, and can rotate continuously around a vertical axis.

[0021] The binocular structured light vision module 4 includes a binocular camera, a structured light projector, and a substrate, which are mounted on a rotating gimbal 3.

[0022] The vehicle-mounted controller 5 is installed on the upper surface of the pipeline inspection robot body 1 and is used to control the height adjustment of the lifting slide 2.

[0023] The vehicle-mounted controller 5 is electrically connected to the lifting slide 2 and the rotating gimbal 3 respectively.

[0024] In this embodiment of the invention, by combining the lifting slide 2, the rotating gimbal 3, and the binocular structured light vision module 4 onto the main body of the pipeline inspection robot and controlling them uniformly by the onboard controller 5, multi-angle, high-precision detection of the pipeline inner wall is achieved. The lifting slide can precisely adjust the height of the vision module to align the camera baseline with the pipeline axis, reducing eccentricity errors. The rotating gimbal 3 allows the vision module to rotate continuously around the vertical axis, covering the circumference of the pipeline. The binocular structured light vision module 4 provides high-precision three-dimensional measurement capabilities. The onboard controller 5 centrally controls all components, ensuring coordinated system movement, improving detection efficiency and data acquisition reliability, and achieving rapid, accurate, and comprehensive detection of defects on the pipeline inner wall.

[0025] In one possible implementation, the lifting slide 2 includes a lead screw guide mechanism, a drive motor, and a slider, which drives the lead screw guide mechanism to rotate in order to control the slider to move vertically up and down.

[0026] In this embodiment of the invention, the lifting slide 2 is equipped with a lead screw guide mechanism, a drive motor and a slider. By rotating the drive screw, the slider moves precisely in the vertical direction, allowing the binocular structured light vision module 4 to adjust its height quickly and stably. This ensures that the camera baseline is aligned with the pipeline axis, reduces eccentricity error, improves the accuracy of three-dimensional measurement and the reliability of detection, while taking into account the stability of the mechanical structure and the flexibility of operation, thus achieving efficient and accurate pipeline inner wall detection.

[0027] In one possible implementation, the rotating gimbal 3 includes a stepper motor and a planetary gearbox, with the stepper motor driving the binocular structured light vision module 4 to rotate around a vertical axis via the planetary gearbox.

[0028] Specifically, the rotation axis of the rotating gimbal 3 is precisely arranged in its structural design to pass through the overall center of gravity of the binocular structured light module (including the binocular camera, structured light projector, and its mounting bracket). This center of gravity balancing design prevents the module from generating additional eccentric torque during rotation, allowing the gimbal drive motor to achieve smooth and precise angular displacement control by overcoming only a small inertial load. With this center of gravity axis arrangement, the angular overshoot and steady-state error of the gimbal during startup, shutdown, and uniform rotation are significantly reduced, resulting in higher attitude consistency among images acquired from multiple angles and thus improving the accuracy of subsequent multi-view point cloud registration. Simultaneously, the reduced torque requirement allows for the use of smaller drive motors and reduction mechanisms, helping to reduce the load on the robot's end effector and improving the overall mobility and endurance within pipelines.

[0029] In this embodiment of the invention, the stepper motor can achieve precise rotation control according to a preset angle, and the planetary gearbox can increase the output torque and reduce the rotation speed, making the binocular structured light vision module 4 rotate more smoothly and reliably around the vertical axis. Through this structure, the vision module can sequentially acquire images of the inner wall in different circumferential directions inside the pipe, expanding the detection coverage, reducing blind spots, and improving the accuracy of multi-angle scanning and 3D reconstruction.

[0030] In one possible implementation, the binocular camera includes a left camera and a right camera, which are symmetrically arranged on both sides of the structured light projector.

[0031] Specifically, the binocular structured light vision module is a manually configurable precision measurement component. Its mechanical structure has multiple degrees of freedom for adjustment and can be pre-set according to the diameter, material and surface characteristics of different pipes before inspection.

[0032] Furthermore, the left and right cameras are mounted together on a one-dimensional rotating gimbal, which can be manually rotated around a vertical axis to change the angle between the optical axis of the binocular cameras and the axis of the pipe. For pipes with smaller diameters, this angle can be increased to make the common field of view of the cameras more focused on the area to be measured on the inner wall of the pipe. For pipes with larger diameters, the angle can be decreased to ensure sufficient field of view width. In addition, the left and right cameras can be manually slid horizontally towards or away from each other on the gimbal, thereby achieving continuous adjustment of the baseline distance. Increasing the baseline is beneficial to improving the absolute accuracy of depth measurement, but it will reduce the common field of view. Shortening the baseline can expand the field of view, but the depth accuracy will be reduced. The operator can pre-determine the optimal baseline distance before inspection based on the actual pipe size and the required inspection accuracy.

[0033] In this embodiment of the invention, the left and right cameras are symmetrically positioned on either side of the structured light projector. This facilitates a more balanced imaging relationship of the structured light pattern within the fields of view of the left and right cameras, reducing matching errors caused by skewed projection positions. Simultaneously, the binocular cameras can simultaneously acquire images of the pipe's inner wall from two perspectives, combining this with structured light projection information for corresponding point matching and 3D reconstruction. This improves measurement stability and reconstruction accuracy in low-texture, low-light pipe environments.

[0034] In one possible implementation, the optical center of the structured light projector is located on the perpendicular plane of the line connecting the optical centers of the left and right cameras.

[0035] Specifically, the structured light projector is mounted in the center of the binocular cameras, and its support has a sliding mechanism that allows the projector to be manually moved back and forth relative to the camera plane along the optical axis. More importantly, the projector's optical center is precisely mechanically positioned both horizontally and vertically, ensuring it always lies on the perpendicular bisector of the line connecting the optical centers of the left and right cameras—that is, the projector's optical center is strictly aligned with the geometric center of the two cameras. This fixed layout with the projector's optical center in the center guarantees that the coded structured light pattern projected onto the inner wall of the pipe has complete symmetry with respect to the left and right cameras. This effectively eliminates phase distortion caused by asymmetric projection angles and inherent deviations in left-right image matching, significantly improving the robustness of stereo matching and the accuracy of 3D reconstruction.

[0036] Furthermore, all adjustments (pan-tilt head angle, baseline distance, projector position) are manually completed by the operator in one go before the inspection begins, based on the actual operating conditions of the pipeline to be inspected. Once the adjustments are complete, all moving parts are locked in place, preventing any relative movement during the entire pipeline inspection process. This "manual adaptation first, then locking and inspecting" design ensures the system's adaptability to different pipe diameters while avoiding additional errors and unreliability caused by mechanical adjustments during the inspection process. It is particularly suitable for automated inspection tasks of batch, long-distance, and uniformly sized pipelines.

[0037] In this embodiment of the invention, the optical center of the structured light projector is positioned on the perpendicular plane of the line connecting the optical centers of the left and right cameras. This ensures that the projected structured light pattern is symmetrically imaged in the fields of view of the left and right cameras, thereby reducing stereo matching errors caused by projection eccentricity. This layout improves the matching accuracy and 3D reconstruction accuracy of binocular structured light stereo vision, making the measurement of the inner wall of the pipe more accurate and reliable, especially ensuring stable defect detection results in environments with weak texture and low light.

[0038] In one possible implementation, the center of gravity of the binocular structured light vision module 4 is located on the rotation axis of the rotating gimbal 3.

[0039] In this embodiment of the invention, the center of gravity of the binocular structured light vision module 4 is set on the rotation axis of the rotating gimbal. This reduces the eccentric torque generated when the vision module rotates, making the rotating gimbal 3 more stable during startup, shutdown, and rotation. This structure reduces the load on the stepper motor and planetary gearbox, improves the accuracy of rotation angle control, and reduces jitter during image acquisition, thereby enhancing the stability of multi-angle scanning and the accuracy of three-dimensional reconstruction of the pipe inner wall.

[0040] In one possible implementation, the vehicle controller 5 employs an embedded artificial intelligence processing platform with an integrated neural network processing unit.

[0041] Specifically, the vehicle controller uses the embedded AI platform RK3588, which integrates an NPU to accelerate deep learning inference.

[0042] In this embodiment of the invention, the vehicle-mounted controller 5 employs an embedded artificial intelligence processing platform with an integrated neural network processing unit, enabling direct image processing and defect identification of the pipeline inner wall at the pipeline inspection robot body 1. The neural network processing unit provides hardware acceleration for the lightweight semantic segmentation network, improving image inference speed and reducing reliance on the host computer and wireless communication. This reduces the bandwidth pressure caused by the large amount of raw image backhaul, thereby improving the system's real-time detection capability and field adaptability.

[0043] In one possible implementation, the vehicle-mounted controller 5 is used to control the binocular camera to measure distances in the circumferential direction of the pipe, and to calculate the pipe diameter based on the distance measurement results.

[0044] The vehicle-mounted controller is used to control the movement of the lifting slide according to the diameter, so that the center height of the binocular structured light vision module 4 is consistent with the height of the pipe axis.

[0045] In this embodiment of the invention, the vehicle-mounted controller 5 controls a binocular camera to measure the distance along the circumference of the pipe, calculates the pipe diameter based on the measurement results, and then controls the lifting slide to adjust the installation height of the binocular structured light vision module 4 so that its center height is consistent with the pipe axis height. This solution can effectively reduce the eccentricity error between the vision module and the pipe center, improve the accuracy of subsequent structured light scanning, stereo matching, and 3D reconstruction, thereby improving the accuracy and reliability of the pipe inner wall defect detection results.

[0046] In one possible implementation, the on-board controller 5 is used to segment defects in the acquired images of the inner wall of the pipe.

[0047] A three-dimensional point cloud model of the inner wall of the pipe is generated using the principle of binocular structured light stereo vision.

[0048] The vehicle-mounted controller 5 is used to mark defect areas in the 3D point cloud model.

[0049] In this embodiment of the invention, the vehicle-mounted controller performs defect segmentation on the acquired images of the inner wall of the pipeline and generates a three-dimensional point cloud model using the principle of binocular structured light stereo vision, thereby mapping the two-dimensional defect information into three-dimensional space. By marking the defect areas in the three-dimensional point cloud model, the visual location of defects on the inner wall of the pipeline can be realized, intuitively displaying the location and shape of the defects. This not only improves the accuracy and reliability of detection but also facilitates subsequent pipeline maintenance and risk assessment.

[0050] Reference manual attached Figure 2 The diagram shows a flowchart of a robot method for inspecting the inner wall of a pipeline provided by an embodiment of the present invention.

[0051] S1: Place the pipeline inspection robot body 1 in the pipeline to be inspected, and measure the distance from the inner wall of the pipeline to the binocular structured light vision module 4 through the binocular structured light vision module 4.

[0052] Specifically, the pipeline inspection robot is placed inside the pipeline at the inlet. The onboard controller 5 controls the binocular camera to rotate sequentially to 0 degrees and 180 degrees. Using the principle of binocular parallax, the distance from the inner wall of the pipeline to the camera in each direction is measured. The sum of the two distances, plus twice the distance from the camera's optical center to the rotation axis, is used as an estimate of the pipeline diameter.

[0053] In this embodiment of the invention, the pipeline inspection robot body 1 is placed inside the pipeline to be inspected, and the distance to the inner wall is measured by the binocular structured light vision module 4. This allows for the rapid acquisition of pipeline geometric information, providing accurate reference for subsequent pipeline diameter estimation, lifting and centering, and multi-angle scanning. It also ensures that the baseline of the vision module is aligned with the pipeline axis, thereby improving the accuracy of 3D reconstruction and the reliability of defect detection.

[0054] S2: Based on the distance, the diameter of the pipe to be inspected is estimated by the vehicle controller 5, and the inner wall radius of the pipe is determined based on the pipe diameter.

[0055] In this embodiment of the invention, the vehicle controller 5 estimates the diameter of the pipe to be inspected based on the distance information measured by the binocular structured light vision module 4, and further determines the inner wall radius of the pipe. This enables accurate acquisition of the geometric dimension parameters of the pipe, providing basic data support for subsequent height adjustment of the vision module, constraint of the search range of corresponding points, and three-dimensional reconstruction process, thereby improving the accuracy and reliability of pipe inner wall detection and defect location.

[0056] S3: Based on the inner wall radius of the pipe, control the lifting slide 2 to move the binocular structured light vision module 4 in the vertical direction, so that the binocular structured light vision module 4 is raised to the axial height of the pipe to be inspected.

[0057] Specifically, the vehicle controller 5 drives the lifting slide to raise the binocular structured light vision module 4 to the height of the pipe axis and maintains this height throughout the subsequent inspection process, so that the rotation axis of the rotating gimbal 3 coincides with the central axis of the pipe, eliminating the measurement error caused by camera eccentricity.

[0058] In this embodiment of the invention, based on the inner wall radius of the pipe, the lifting slide 2 is controlled to move the binocular structured light vision module 4 vertically, aligning its center height with the pipe axis. This operation eliminates the eccentricity error caused by height deviation of the vision module, ensuring that the baseline of structured light scanning and binocular stereo vision measurement coincides with the pipe axis, thereby improving the accuracy and reliability of subsequent multi-angle scanning, 3D reconstruction, and defect detection, while maintaining the stability of the vision module inside the pipe.

[0059] S4: The on-board controller 5 controls the pipeline inspection robot body 1 to move along the pipeline axis according to the preset step length, and the on-board controller 5 controls the rotating gimbal 3 to drive the binocular structured light vision module 4 to rotate to multiple preset angles in sequence.

[0060] Specifically, the onboard controller controls the pipeline inspection robot to move forward in predetermined steps. After moving a predetermined distance, the robot stops and then controls the rotating gimbal to rotate the binocular structured light vision module to a predetermined angle.

[0061] In this embodiment of the invention, the onboard controller uniformly controls the pipeline inspection robot to move along the pipeline axis at preset step lengths, and controls the rotating gimbal to rotate the binocular structured light vision module to multiple preset angles, enabling segmented scanning of the pipeline in both axial and circumferential directions. This method can expand the detection coverage of the pipeline inner wall, reduce blind spots, and ensure that images acquired at different positions and angles have good sequence and correlation, thereby providing a stable and reliable data foundation for subsequent image acquisition, point cloud fusion, and defect localization.

[0062] S5: At various preset angles, the vehicle-mounted controller 5 controls the structured light projector to project Gray code stripes and phase-shift stripes onto the inner wall of the pipe to be inspected, and controls the binocular camera to simultaneously acquire images of the inner wall of the pipe.

[0063] Specifically, in each direction, the structured light projector actively projects Gray code fringes and phase-shift fringes, while the binocular camera simultaneously acquires left and right images. The acquired images are directly received by the vehicle-mounted controller.

[0064] It should be noted that those skilled in the art can set the size of the preset angle according to actual needs, and this invention does not limit it.

[0065] In this embodiment of the invention, the vehicle-mounted controller 5 controls the structured light projector to project Gray code fringes and phase-shift fringes at each preset angle, and controls the binocular camera to synchronously acquire images of the inner wall of the pipe, thereby obtaining left and right images with structured light encoded information. By actively projecting structured light, the matching features of the inner wall of the pipe in weak texture and low-light environments can be enhanced, improving the accuracy of subsequent phase calculation, corresponding point matching, and 3D reconstruction. At the same time, synchronous acquisition also helps to ensure the consistency of the left and right image data.

[0066] S6: Input the image of the inner wall of the pipe into a lightweight semantic segmentation network, output a pixel-level defect mask, and extract the set of defect pixel coordinates from the pixel-level defect mask.

[0067] The lightweight semantic segmentation network uses MobileNetV2 as its backbone feature extraction network, combined with the Deeplabv3+ segmentation framework. MobileNetV2 employs an inverse residual structure and depthwise separable convolutions, significantly reducing the number of parameters and computational cost.

[0068] Specifically, for input size is The images of the inner wall of the pipe, and the sizes of the feature maps output by the backbone network are as follows: , , Different scales are used, with high-level feature maps containing rich semantic information and shallow feature maps preserving fine spatial details. The high-level features output by the backbone network (size...) The data is fed into the ASPP module. The ASPP module consists of multiple dilated convolutions in parallel with different dilation rates, r=6, 12, 18, and the receptive field size of the dilated convolution kernel is:

[0069] in, k This indicates the kernel size (3×3). r Indicates the expansion rate. ReceptiveField This indicates the size of the receptive field of the dilated convolution kernel.

[0070] By using dilated convolutions with different dilation rates, ASPP can acquire contextual information at different scales without increasing resolution loss, thereby improving the ability to identify defect regions of different sizes. The features output by ASPP are then subjected to dimensionality reduction through convolution before entering the Decoder branch. The Decoder first upsamples the ASPP results by a factor of 4, obtaining a size of... The feature map is then concatenated and fused with the shallow features of the corresponding layer (layer 3 output) in MobileNetV2. The fused features are then subjected to two 3×3 convolutions and one 4x upsampling, and the final output is the same size as the original image. H × WSame pixel-level defect mask ,in, This indicates that the pixel belongs to a defect area. This indicates a normal region. The vehicle controller uses the RK3588 embedded AI platform, converting the trained network model into RKNN format for deployment. During inference, the input image tensor size is 3× H × W The RK3588's integrated NPU provides hardware acceleration for convolution operations, achieving a measured inference time of less than 50 milliseconds per frame, meeting real-time inspection requirements. After segmentation, data is extracted from the mask... Extract the coordinate set of all defective pixels:

[0071] in, The set of coordinates representing defective pixels. x i Indicates the first i The x-coordinate of each defective pixel in the image. y i Indicates the first i The vertical coordinate of each defective pixel in the image. This indicates that the pixel-level defect mask is applied to the defect pixel. x i , y i The value at ) N This represents the total number of defective pixels. This set of coordinates will be passed to the 3D reconstruction module for subsequent color marking of the defective areas.

[0072] In this embodiment of the invention, the acquired image of the pipe's inner wall is input into a lightweight semantic segmentation network, enabling pixel-level defect recognition and outputting a defect mask. By extracting the set of defect pixel coordinates from the mask, defects in the two-dimensional image can be accurately located, providing fundamental data for subsequent absolute phase calculation, 3D point cloud reconstruction, and 3D defect labeling. This step, by processing the image in real time at the edge, reduces reliance on image backhaul and remote computation, improving detection efficiency and real-time performance while ensuring the accuracy and reliability of defect recognition.

[0073] S7: Calculate the wrapping phase based on the inner wall image using a four-step phase shift formula.

[0074] Specifically, a structured light coding strategy combining a four-step phase-shifting method and Gray code coding is adopted. The four-step phase-shifting method projects four phases that are sequentially phase-differentiated. The sinusoidal fringe pattern, the grayscale distribution of the four images can be represented as:

[0075]

[0076]

[0077]

[0078] in, This represents the grayscale distribution of the first image. This represents the grayscale distribution of the second image. This represents the grayscale distribution of the third image. This represents the grayscale distribution of the fourth image. Represents defective pixels ( x , y Background light intensity distribution at location ) Represents defective pixels ( x , y Modulated light intensity at ) Represents defective pixels ( x , y The wrapping phase at ) π It represents pi (π).

[0079] By combining the above four equations, the wrapped phase can be determined. :

[0080] in, arctan This represents the arctangent function.

[0081] In this embodiment of the invention, the wrapping phase of each pixel can be calculated using the grayscale values ​​of four images, providing fundamental information for subsequent absolute phase calculation and 3D reconstruction. The advantage of this step is that it accurately reflects the geometric features of the pipe's inner wall, improving the accuracy of stereo matching and 3D reconstruction in weakly textured environments.

[0082] S8: Determine whether the package phase is within the preset phase range. If so, the package phase is the absolute phase. Otherwise, decode the inner wall image through the vehicle controller 5 to obtain the phase period order, and calculate the absolute phase based on the phase period order.

[0083] Specifically, since the range of the arctangent function is The above The phase is truncated within this interval, appearing as a discontinuous, jagged shape, known as the wrapped phase. To obtain a continuous absolute phase, this embodiment further projects a Gray code sequence. Assuming a K-bit Gray code is projected, the phase period order of each pixel is... k ( x,y (This can be obtained by decoding the Gray code image.) Absolute phase. Phase with package The relationship is:

[0084] in, k ( x,y This indicates the phase period order of each pixel. Indicates absolute phase. Absolute phase It is monotonically increasing and unique throughout the entire measurement range, thus serving as a reliable primitive for matching corresponding points between left and right camera images.

[0085] In this embodiment of the invention, the absolute phase can be quickly determined by judging whether the wrapped phase is within a preset phase range. For wrapped phases that are outside the range, the on-board controller calculates the absolute phase by decoding the phase period order, thereby correcting the phase period jump. The advantage of this step is that it can accurately obtain the absolute phase information of each pixel, ensuring the accuracy of matching corresponding points in subsequent left and right images, thereby improving the accuracy of 3D reconstruction and the reliability of locating defects in the inner wall of the pipe, and can work stably even in complex textures or low-light environments.

[0086] S9: Based on the absolute phase and the inner radius of the pipe wall, limit the search range of the inner wall image of the pipe to determine the corresponding point pairs in the inner wall image of the pipe.

[0087] Specifically, in the stereo matching stage, stereo calibration is first performed on the left and right cameras to align the epipolar lines horizontally. For any pixel in the left image... Its absolute phase value is In the right image, search for pixels that satisfy the following conditions. :

[0088] in, The x-coordinate is x r The vertical axis is y r The absolute phase value of a pixel represents, The x-coordinate is x l The vertical axis is y l The absolute phase value of the pixel, x r The x-coordinate is r pixels, y r The vertical axis is r pixels, x l The x-coordinate is l pixels, y l The vertical axis isl The pixels.

[0089] This involves searching for points that have the same absolute phase value as the pixels in the left image and lie on the same horizontal epipolar line. In actual processing, due to quantization errors and noise, the search focuses on finding the point with the smallest absolute phase difference. To accelerate the matching process, this invention introduces the geometric characteristics of a cylindrical pipe. Let the inner wall radius of the pipe be... R The baseline distance of the binocular cameras is B Then the search range for corresponding points in the right image can be limited to the following interval:

[0090] in, This represents the maximum parallax determined by the pipe diameter and camera parameters. Let denot be the minimum parallax determined by the pipe diameter and camera parameters, and ∪ denote the union.

[0091] Calculate using the relationship between similar triangles:

[0092] in, f Indicates the camera's focal length. B Indicates the baseline distance of the binocular cameras. R Indicates the inner radius of the pipe. This represents the estimated maximum value of the pipe's inner wall radius. This represents the estimated minimum value of the pipe's inner wall radius. Δ This indicates the allowable depth fluctuation margin, used to appropriately widen the search range to account for measurement errors or pipe surface irregularities.

[0093] By compressing the search range from the entire epipolar line, the width W is reduced to a local interval (approximately). Matching speed can be increased several times.

[0094] In this embodiment of the invention, the search range of corresponding points in the image is limited by using absolute phase combined with the inner wall radius of the pipe. This effectively reduces the search area for matching left and right images, avoiding the large computational load and matching errors caused by full-image search. The advantage of this step is that it improves the efficiency and accuracy of corresponding point matching, providing a reliable foundation for subsequent disparity calculation and 3D point cloud generation. At the same time, it can maintain matching stability and 3D reconstruction accuracy in environments with insufficient lighting and poor texture inside the pipe.

[0095] S10: Based on the same point pairs, calculate the three-dimensional point cloud coordinates using the principle of binocular visual triangulation, and generate a three-dimensional point cloud model of the inner wall of the pipe based on the three-dimensional point cloud coordinates.

[0096] Specifically, obtain pairs of points with the same name. and After, parallaxd for:

[0097] in, d Indicates parallax.

[0098] Based on the triangulation principle of binocular vision, the three-dimensional coordinates of spatial point P are... It can be calculated using the following formula:

[0099]

[0100]

[0101] in, X This represents the x-coordinate of a point in the camera coordinate system. Y This represents the ordinate of a point in space within the camera coordinate system. Z This represents the depth distance from a point in space to the camera plane. Indicates the x-coordinate of the camera's principal point. This represents the ordinate of the camera's principal point.

[0102] After performing the above matching and calculation on every pixel in the left and right images, dense point cloud data from a single viewpoint can be obtained. The point clouds collected by the rotating gimbal at different angles are then transformed by coordinates and precisely registered using ICP, and finally fused to generate a complete 3D point cloud model of the pipe's inner wall.

[0103] In this embodiment of the invention, by calculating the coordinates of a three-dimensional point cloud and generating a three-dimensional point cloud model of the pipe's inner wall, two-dimensional image information can be transformed into a three-dimensional structure in real space, achieving accurate reconstruction of the geometric shape of the pipe's inner wall. The advantage of this step is that it provides high-precision three-dimensional data, offering a reliable spatial information foundation for defect location, morphological analysis, and subsequent pipeline maintenance decisions.

[0104] S11: Input the set of defect pixel coordinates into the 3D point cloud model of the inner wall of the pipe, and output the defect area of ​​the pipe to be detected.

[0105] Specifically, based on the set of defect pixel coordinates, the corresponding spatial points are located in the 3D point cloud model, and the color attribute of these points is set to red, thereby realizing the visual marking of the defect area.

[0106] In this embodiment of the invention, mapping the previously extracted set of defect pixel coordinates to a three-dimensional point cloud model of the pipeline inner wall enables the correspondence between defect information in the two-dimensional image and its three-dimensional spatial location. The advantage of this operation is that it can intuitively mark the three-dimensional location and shape of defects on the pipeline inner wall, making the detection results more visual, easier to analyze and maintain, and simultaneously improving the accuracy of defect location and the overall reliability of the detection system, which is beneficial for subsequent pipeline safety assessments and maintenance decisions.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A robotic system for inspecting the inner wall of a pipeline, characterized in that, include: Pipeline inspection robot body (1), lifting slide (2), rotating gimbal (3), binocular structured light vision module (4), vehicle controller (5); The lifting slide (2) is installed on the upper surface of the pipeline inspection robot body (1); The rotating gimbal (3) is installed on the top of the lifting slide (2), and the base of the rotating gimbal (3) is fixedly connected to the moving platform of the lifting slide (2) by bolts. The rotating gimbal (3) includes a stepper motor and a planetary gearbox, and can rotate continuously around a vertical axis; The binocular structured light vision module (4) includes a binocular camera, a structured light projector and a substrate, which are mounted on the rotating gimbal (3); The vehicle controller (5) is installed on the upper surface of the pipeline inspection robot body (1) and is used to control the height adjustment of the lifting slide (2); The vehicle controller (5) is electrically connected to the lifting slide (2) and the rotating gimbal (3) respectively.

2. The pipeline inner wall inspection robot system according to claim 1, characterized in that, The lifting slide (2) includes a lead screw guide mechanism, a drive motor and a slider, which are used to drive the lead screw guide mechanism to rotate so as to control the slider to move up and down vertically.

3. The pipeline inner wall inspection robot system according to claim 1, characterized in that, The rotating gimbal (3) includes a stepper motor and a planetary gearbox. The stepper motor drives the binocular structured light vision module (4) to rotate around the vertical axis via the planetary gearbox.

4. The pipeline inner wall inspection robot system according to claim 1, characterized in that, The binocular camera includes a left camera and a right camera, which are symmetrically arranged on both sides of the structured light projector.

5. The pipeline inner wall inspection robot system according to claim 1, characterized in that, The optical center of the structured light projector is located on the perpendicular plane of the line connecting the optical centers of the left camera and the right camera.

6. The pipeline inner wall inspection robot system according to claim 1, characterized in that, The center of gravity of the binocular structured light vision module (4) is located on the rotation axis of the rotating gimbal (3).

7. The pipeline inner wall inspection robot system according to claim 1, characterized in that, The vehicle controller (5) adopts an embedded artificial intelligence processing platform with an integrated neural network processing unit.

8. The pipeline inner wall inspection robot system according to claim 1, characterized in that, The vehicle controller (5) is used to control the binocular camera to measure distance in the direction of the pipe circumference, and to calculate the pipe diameter based on the measurement result; The vehicle controller is used to control the movement of the lifting slide according to the diameter, so that the center height of the binocular structured light vision module (4) is consistent with the height of the pipeline axis.

9. The pipeline inner wall inspection robot system according to claim 1, characterized in that, The vehicle-mounted controller (5) is used to segment defects in the acquired images of the inner wall of the pipe. A three-dimensional point cloud model of the inner wall of the pipe is generated using the principle of binocular structured light stereo vision. The vehicle controller (5) is used to mark the defect areas in the three-dimensional point cloud model.

10. A method for inspecting the inner wall of a pipe using a robot, characterized in that, Applied to the pipe inner wall inspection robot system according to any one of claims 1 to 9; the method includes: S1: Place the pipeline inspection robot body (1) in the pipeline to be inspected, and measure the distance from the inner wall of the pipeline to be inspected to the binocular structured light vision module (4) through the binocular structured light vision module (4); S2: Based on the distance, the pipe diameter of the pipe to be detected is estimated by the vehicle controller (5), and the inner wall radius of the pipe is determined based on the pipe diameter; S3: Based on the inner wall radius of the pipe, control the lifting slide (2) to drive the binocular structured light vision module (4) to move in the vertical direction, so that the binocular structured light vision module (4) is raised to the axial height of the pipe to be inspected; S4: The vehicle controller (5) controls the pipeline inspection robot body (1) to move along the pipeline axis according to the preset step length, and the vehicle controller (5) controls the rotating gimbal (3) to drive the binocular structured light vision module (4) to rotate to multiple preset angles in sequence. S5: At each of the preset angles, the vehicle controller (5) controls the structured light projector to project Gray code stripes and phase shift stripes onto the inner wall of the pipe to be inspected, and controls the binocular camera to simultaneously acquire images of the inner wall of the pipe. S6: Input the image of the inner wall of the pipe into a lightweight semantic segmentation network, output a pixel-level defect mask, and extract the set of defect pixel coordinates in the pixel-level defect mask; S7: Calculate the wrapping phase based on the image of the inner wall of the pipe using a four-step phase shift formula; S8: Determine whether the packaged phase is within the preset phase range. If so, the packaged phase is the absolute phase. Otherwise, decode the image of the inner wall of the pipe through the vehicle controller (5) to obtain the phase period order, and calculate the absolute phase based on the phase period order. S9: Based on the absolute phase and the inner wall radius of the pipe, limit the search range of the inner wall image of the pipe to determine the corresponding point pairs in the inner wall image of the pipe; S10: Based on the corresponding point pairs, calculate the three-dimensional point cloud coordinates using the binocular vision triangulation principle, and generate a three-dimensional point cloud model of the inner wall of the pipe based on the three-dimensional point cloud coordinates. S11: Input the set of defect pixel coordinates into the three-dimensional point cloud model of the inner wall of the pipeline, and output the defect area of ​​the pipeline to be detected.

Images