A pipeline robot multi-source perception full-true simulation test system based on spatial sampling consistency determination

Abstract

The application discloses a kind of based on space sampling consistency determination pipe robot multi-source perception full true simulation test system, belong to pipe detection simulation test technical field, it includes mechanical execution system, space state perception gate and physical trigger system, multi-sensor sampling unit, true value reconstruction system.Mechanical execution system provides the motion reference along the axial direction of pipeline, and allow to introduce controlled nonlinear disturbance, space state perception gate and physical trigger system are used for the arbitration of sampling opportunity when FPGA motion control card, step modulation and physical trigger;Multi-sensor sampling unit is used to execute sampling instruction and generate original data with position label;True value reconstruction system is used to construct the standardized pipeline true value containing position, texture and geometric information;The application realizes high-fidelity three-dimensional digital reconstruction to the inner wall of pipeline by introducing IMU posture latch, micro time delay elastic capture, binocular vision forward variable frequency modulation and reverse geometry correction algorithm.

Description

Technical Field

[0001] This invention belongs to the field of pipeline inspection simulation test technology, specifically involving a pipeline robot multi-source perception full-scale simulation test system based on spatial sampling consistency judgment. Background Technology

[0002] Long-distance oil and gas pipelines serve as strategic "main arteries" for national energy transportation, and their structural integrity is directly related to national energy security and ecological environment security. However, these pipelines operate under harsh environments, including underground burial, high-pressure transmission, and complex geological changes, making them highly susceptible to structural damage such as electrochemical corrosion, stress corrosion cracking, and mechanical deformation. Due to the inaccessibility of the external environment, using pipeline inspection robots equipped with high-precision sensors to enter the pipeline and conduct online, full-coverage, non-destructive inspections has become a primary technical means for assessing pipeline health and preventing catastrophic leaks.

[0003] However, in practical engineering applications, robots operate in enclosed, slippery, and unstructured complex pipe environments, often exhibiting unstable motion characteristics such as wheel slippage, slight vibrations, and nonlinear speed changes due to nonholonomic constraints. Developing and validating multi-source perception algorithms (such as SLAM and 3D reconstruction) directly in real pipes not only incurs extremely high trial-and-error costs but also makes it difficult to quantitatively evaluate the cumulative errors of the algorithms due to the lack of high-precision absolute position references. Therefore, there is an urgent need to construct a "full-scale simulation test system" capable of accurately reproducing the kinematic characteristics of real robots and providing full-element ground truth data in a laboratory environment.

[0004] Existing simulation testing technologies suffer from the following key challenges: Lack of rigid alignment of multi-source data: Traditional solutions often rely on system timestamps for "soft synchronization" (such as CN121043191A), which cannot solve the spatial registration problem of heterogeneous sensors (such as array cameras and binocular vision modules) in variable-speed motion. When the motion speed fluctuates, the physical spacing corresponding to the same time interval is not constant, resulting in uneven spatial sampling density.

[0005] The impact of dynamic attitude on imaging is ignored: existing test benches mostly assume that the motion is stable, using "position arrival" as the sole triggering condition. However, in real pipeline conditions, instantaneous vibrations are inevitable when the robot crosses obstacles. Existing systems lack the ability to simulate, sense, and gate these vibrations, which may lead to "blind triggering" at the vibration peak, directly causing imaging axis shift, image blurring, and point cloud layering.

[0006] Lack of high-fidelity ground truth with full elements: Existing systems typically only provide ground truth in a single dimension (image only or depth only), lacking the ability to fuse "macro-geometric deformations (perceived by binoculars)" with "micro-texture details (perceived by array cameras)" at the pixel level. In addition, existing technologies have failed to adequately address the data spatial misalignment problem caused by the physical installation spacing between the front-end sensing module and the circumferential array camera, making it impossible to construct a seamless, full-element 3D model.

[0007] The sampling strategy is rigid and lacks environmental adaptability: Existing test benches generally use "fixed step size triggering" (e.g., taking one picture every 1 mm). This "rigid" strategy cannot perceive the density distribution of features on the inner wall of the pipe. When encountering key feature areas such as welds, corrosion pits, and diameter changes, the fixed step size often leads to insufficient sampling density and loss of high-frequency details; while in straight pipe sections, excessively high sampling rates cause data redundancy. Existing technologies fail to utilize the "pathfinding" advantages of front-end sensors (such as binocular vision) and cannot achieve "predictive encrypted sampling" of key features; at the same time, when real robots pass through welds, they encounter mechanical vibrations that accompany obstacle crossing. The high-frequency triggering requirements and the vibration damping mechanism (IMU steady-state constraints) will create irreconcilable timing conflicts, resulting in a large amount of valid data being gated and intercepted due to "excessive vibration". Summary of the Invention

[0008] To address the aforementioned problems, this invention provides a multi-source perception full-scale simulation test system for pipeline robots based on spatial sampling consistency determination, thus solving the problem of low data reliability in simulation test technology.

[0009] A multi-source perception full-scale simulation test system for a pipeline robot based on spatial sampling consistency determination includes a mechanical execution system, a spatial state perception gating and physical triggering system, a multi-sensor sampling unit, and a truth reconstruction system. The mechanical execution system includes a cylindrical positioning support frame on which a cylindrical pipe is mounted. A telescopic mechanism is provided next to the cylindrical positioning support frame. The telescopic mechanism includes a bottom frame, an electrical control box is provided on the side of the bottom frame near the cylindrical positioning support frame, and a telescopic fork assembly is also provided. The central axis of the telescopic fork assembly is on the same straight line as the axis of the cylindrical pipe. A support rod is provided on the bottom frame, and a linear guide pair for guiding the telescopic fork assembly is horizontally provided at the top of the support rod. One end of the telescopic fork assembly is sleeved in the electrical control box, and the other end of the telescopic fork assembly is provided with a connector. The other end of the connector is locked in the linear guide pair and can slide along the length of the linear guide pair. The electrical control box is provided with a power module and a drive component for driving the axial movement of the telescopic fork assembly. The drive component includes a servo motor, a servo driver, and an FPGA motion control card. The spatial state perception gating and physical triggering system is used by the FPGA motion control card for arbitration of sampling timing, step size modulation and physical triggering; The multi-sensor sampling unit is used to execute sampling instructions and generate raw data with location tags; The truth reconstruction system is used to perform deterministic spatiotemporal alignment of two-dimensional textures and three-dimensional geometry based on physical mileage indexes, and to construct standardized pipeline truth values ​​containing position, texture and geometric information through geometric inverse correction and high-precision texture mapping. The multi-sensor sampling unit includes a multi-sensor module, which is installed on the end face of the telescopic fork assembly that is sleeved on one end of the electrical control box. The top of the electrical control box is equipped with a human-machine interface panel. The human-machine interface panel is electrically connected to the FPGA motion control card, servo motor, power module, and multi-sensor module. The servo motor and multi-sensor module are electrically connected to the FPGA motion control card. The servo motor, FPGA motion control card, multi-sensor module, and power module are electrically connected to the power module. The human-machine interface panel is also connected to a host computer.

[0010] Furthermore, the multi-sensor module includes a housing, on which a plurality of high-definition cameras are arranged in a circumferential array, and two forward-looking cameras are arranged on the end face of the housing near the cylindrical pipe. A structured light projector is arranged between the two forward-looking cameras, and an IMU is arranged inside the housing.

[0011] Furthermore, the specific operation process of the spatial state perception gating and physical triggering system is as follows: S1, Interference Monitoring: The FPGA motion control card reads the three-axis acceleration and three-axis angle feedback from the IMU in real time, and determines whether the current acceleration or angle deviation exceeds the safety threshold based on the preset safety threshold. If so, it determines that the multi-sensor sampling unit will not perform sampling. S2, Step-size modulation based on parallax perception: By using two forward-looking cameras, an inverse coupling relationship between sampling density and feed rate is established, based on the vertical field of view height of a single high-definition camera set in a circumferential array. and preset image overlap rate Calculate the maximum allowable basic step size. , in, This ensures seamless coverage of the circumferential panoramic image under standard acquisition mode. The FPGA motion control card calculates the parallax change rate of the two forward-looking camera ROI regions in real time. When the parallax d of the two forward-looking cameras stabilizes, the system executes the basic step size. When a parallax step is detected, the FPGA motion control card immediately switches the trigger step size to a fine step size and controls the servo motor to decelerate. S3, Physical Gated Arbitration: When the motor encoder pulse count reaches the sampling step size, a sampling request is generated. The FPGA motion control card checks the current IMU status: if the IMU value is lower than the safety threshold, it immediately triggers the photo capture; if the IMU value is higher than the safety threshold, the FPGA motion control card refuses to trigger immediately and enters S4 micro-delay elastic capture; when the vibration wave passes and the IMU value falls back below the threshold, the trigger signal is resent. S4, micro-delay elastic capture: The system does not skip the current sampling point and actively introduces a dynamic Δt micro-delay. The FPGA motion control card tracks the IMU value in real time. When the IMU value falls below the threshold, a physical trigger signal is sent again. S5, a source-level hard implantation of spacetime: In the same clock cycle as the release trigger signal, the FPGA motion control card extracts the current absolute physical mileage and instantaneous attitude quaternion in parallel, and combines the absolute physical mileage and instantaneous attitude quaternion data into a 64-bit spatiotemporal status stamp, which is then hard-embedded into the frame header of the original data of the multi-sensor sampling unit.

[0012] Furthermore, the specific operation process of the truth reconstruction system is as follows: Step 1, Reverse geometric correction based on status stamp: By utilizing the instantaneous attitude quaternions hard-embedded in the FPGA motion control card in the data frame header, an inverse rotation matrix is ​​constructed to forcibly correct the perspective distortion caused by the vibration of the telescopic fork mechanism to the ideal kinematic coordinate system. Specifically: Parse the quaternion latched at time t in the frame header Convert it into an instantaneous rotation matrix Let K be the camera intrinsic parameter of the array vision unit, and construct the homography transformation matrix from the current jitter pose correction to the ideal horizontal pose. :

[0013] For the original image pixels Perform correction: .

[0014] Step 2, Circular panoramic stitching based on rigid constraints: Using the physical pulse increment of the motor encoder as the sole longitudinal constraint, and combining it with cylindrical projection, rigid splicing without feature points is achieved, specifically as follows: Pre-determine the extrinsic rotation matrix of all high-definition cameras in the circumferential array. Solidify it; for each frame of the original planar image Using camera intrinsic parameters Using the cylindrical projection formula, map it to a unified virtual cylindrical coordinate system. Above, edge perspective stretching caused by camera planar imaging is eliminated, generating a standard number of curved tile images corresponding to the specified number; among them, , ; Based on the external parameters calibrated offline The fixed pixel offset between adjacent cameras in cylindrical coordinates can be calculated directly. The system will connect the right edge of the i-th image to the left edge of the (i+1)-th image. Forced overlap is applied; within a fixed overlap area, a linear weighted average algorithm is used, with the weights gradually changing from 1 to 0 from left to right to smoothly transition the pixel values ​​in the overlap area, eliminating stitching seams caused by lighting differences, and finally outputting a single-frame 360° circular panoramic image. Then, the encoder pulse increment written by the motion control card integrated with the FPGA is read. Calculate the vertical pixel displacement between images :

[0015] in, It is the pulse equivalent; Optical magnification; Pixel size; Based on this, continuous circular panoramic images are stacked at fixed intervals to generate a distortion-free panoramic unfolded image.

[0016] Step 3, Dense 3D Reconstruction Based on Active Structured Light: The 3D coordinates are calculated based on the disparity of binocular vision from two forward-looking cameras. A semi-global matching algorithm is used to calculate the dense disparity map D(u,v) of the corrected left and right images. Based on the speckle texture projected by the structured light projector, matching confidence is ensured on the featureless smooth pipe wall. The triangulation principle is then applied. Where f is the focal length, B is the baseline, and d is the disparity, the disparity map is converted into a depth map; combined with the camera intrinsic parameter K, the depth map is back-projected into a 3D point cloud P(x,y,z); since the depth information Z and texture color (R,G,B) both come from the same optical system, there is no pixel alignment error, and thus a ground truth model with perfect geometry and texture locking can be directly output. Step 4, Spatial Hysteresis Alignment Based on Physical Mileage Index: Based on the inherent physical mounting distance between the two front-view cameras and the high-definition camera array in the circumferential array. The system collects and constructs a 3D geometric slice G1 at physical mileage S1 using two front-facing cameras, labels it with mileage tag S1, and stores it in a cache queue. The system also collects a panoramic texture image T2 at physical mileage S2 using a high-definition camera array, and labels it with mileage tag S2. The system monitors the encoder mileage in real time. When the mileage S2 of the array cameras meets the specified value... At this time, it is determined that the current panoramic texture image T2 corresponds to the previous three-dimensional geometric slice G1, and the texture T2 is accurately mapped onto the geometric model G1.

[0017] Furthermore, the cylindrical positioning support frame includes a base limiting fixture, and a U-shaped support frame is installed on the top of the base limiting fixture.

[0018] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention introduces a spatial state gating mechanism, using an FPGA as a normally closed electronic gate. Before physical data acquisition occurs, the mechanical vibration state is arbitrated. For sampling requests that do not meet stability constraints, the system directly physical intercepts or delays triggering. This ensures from the source that all incoming data is "high-fidelity true value," avoiding the risk of "garbage in, garbage out (GIGO)," while significantly reducing the computational overhead of subsequent algorithms.

[0019] 2. In this invention, a collaborative control chain of "forward-looking perception - active deceleration - encrypted data acquisition" is constructed. When the two forward-looking cameras predict the presence of a diameter change or weld seam ahead, the system actively reduces the motor feed speed while simultaneously increasing the number of sampling points. This strategy effectively prevents timing conflicts between the IMU vibration damping mechanism (which requires stability) and the high-frequency triggering mechanism (which requires speed), ensuring that complete, clear, and high-density ground truth data can still be obtained even at the most complex geometric features.

[0020] 3. This invention exhibits strong environmental adaptability through a "micro-delay elastic capture" mechanism. When the system detects a vibration peak (such as from manual manipulation or a sudden motor stop), it does not trigger blindly but allows the sampling point to undergo controlled displacement at a microscale (e.g., delayed triggering causes a 0.05mm position shift) to actively avoid the moment of maximum vibration energy. Combined with source-level spatiotemporal hard implantation, the FPGA motion control card accurately records the actual physical coordinates after this "slippage." Subsequent algorithms read these precise coordinates for stitching, ensuring both image clarity (avoiding vibration) and seamless stitching (the coordinates are true values). A slight sacrifice in "spatial sampling uniformity" is exchanged for a significant improvement in "imaging quality signal-to-noise ratio."

[0021] 4. In this invention, a high-frequency feature pattern is projected onto the weakly textured inner wall of a pipe using a structured light projector to artificially construct a texture field. This not only enables the system to calculate dense point clouds of millions of pixels even under smooth pipe walls, but also achieves a natural zero-error fusion of geometric structure and texture color because the depth data is directly derived from the image itself, resulting in a significant improvement in the detail richness of the constructed ground truth model. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a schematic diagram of the structure of the present invention; Figure 2 This is a three-dimensional schematic diagram of the present invention; Figure 3 This is a schematic diagram of the structure of the multi-module sensor of the present invention; Figure 4 This is a block diagram illustrating the working principle of the present invention.

[0023] The markings in the diagram are: 1-Cylindrical positioning support frame, 11-Base limiting fixture, 12-U-shaped support frame, 2-Cylindrical pipe, 3-Telescopic mechanism, 31-Bottom frame, 32-Electrical control box, 33-Telescopic fork assembly, 34-Support rod, 35-Linear guide rail pair, 36-Connector, 4-Structured light projector, 5-Housing, 6-High-definition camera, 7-Front-view camera. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0026] It should be noted that the labels and letters in the following figures represent similar items, therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0027] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use. They are only used for the purpose of simplifying the description of this invention and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. In addition, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] Furthermore, terms such as "horizontal" and "vertical" do not imply that components must be absolutely horizontal or suspended, but rather that they can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0029] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0030] Refer to the instruction manual. Figure 1-4 : A multi-source perception full-scale simulation test system for a pipeline robot based on spatial sampling consistency determination includes a mechanical execution system, a spatial state perception gating and physical triggering system, a multi-sensor sampling unit, and a truth reconstruction system. The mechanical execution system includes a cylindrical positioning support frame on which a cylindrical pipe is mounted. A telescopic mechanism is provided next to the cylindrical positioning support frame. The telescopic mechanism includes a bottom frame, an electrical control box is provided on the side of the bottom frame near the cylindrical positioning support frame, and a telescopic fork assembly is also provided. The central axis of the telescopic fork assembly is on the same straight line as the axis of the cylindrical pipe. A support rod is provided on the bottom frame, and a linear guide pair for guiding the telescopic fork assembly is horizontally provided at the top of the support rod. One end of the telescopic fork assembly is sleeved in the electrical control box, and the other end of the telescopic fork assembly is provided with a connector. The other end of the connector is locked in the linear guide pair and can slide along the length of the linear guide pair. The electrical control box is provided with a power module and a drive component for driving the axial movement of the telescopic fork assembly. The drive component includes a servo motor, a servo driver, and an FPGA motion control card. The spatial state perception gating and physical triggering system is used by the FPGA motion control card for arbitration of sampling timing, step size modulation and physical triggering; The multi-sensor sampling unit is used to execute sampling instructions and generate raw data with location tags; The truth reconstruction system is used to perform deterministic spatiotemporal alignment of two-dimensional textures and three-dimensional geometry based on physical mileage indexes, and to construct a standardized pipeline truth containing position, texture and geometric information through geometric inverse correction and high-precision texture mapping. The multi-sensor sampling unit includes a multi-sensor module, which is installed on the end face of the telescopic fork assembly sleeved on one end of the electric control box. The top of the electrical control box is equipped with a human-machine interface panel. The human-machine interface panel is electrically connected to the FPGA motion control card, servo motor, power module, and multi-sensor module. The servo motor and multi-sensor module are electrically connected to the FPGA motion control card. The servo motor, FPGA motion control card, multi-sensor module, and power module are electrically connected to the power module. The human-machine interface panel is also connected to a host computer.

[0031] Specifically, the telescopic fork mechanism, serving as the load transmission medium, employs a multi-stage nested design. Internally, it integrates a sprocket / rack and pinion differential transmission assembly. Through speed ratio design (such as...) This design achieves a significantly increased axial extension stroke with a shorter initial installation length. High-strength aerospace-grade aluminum alloy is used, featuring an I-beam cross-section design optimized by finite element analysis. Combined with a pre-tensioned linear guide pair (eliminating clearances), it ensures controllable end-arm deflection at the maximum extension stroke (e.g., 2 meters). Through servo motor drive, the rotational power of the servo motor is converted into a multi-sensor module's linear feed motion along the pipe axis at double the speed.

[0032] Specifically, the electrical control box integrates a servo motor, servo driver, FPGA motion control card, and power management module. A human-machine interface panel is located on the top of the box for real-time display of motion status and manual control.

[0033] Specifically, the cylindrical positioning support frame includes a base limiting fixture, on the top of which a U-shaped support frame is mounted. Different models of support frames can be used to adapt to cylindrical pipes of different diameters. Additionally, the base limiting fixture, which contacts the ground, can assist in adjusting the height of this support frame to ensure that the center of the pipe coincides with the axis of motion.

[0034] Specifically, the multi-sensor module has connection holes on its back, allowing for rigid mounting to the end of the telescopic fork mechanism. The module body adopts a polyhedral structure and integrates: Arrayed visual unit: namely, 8 high-definition cameras distributed circumferentially, used to capture 360° panoramic texture; Active binocular depth sensing module: Installed at the front of the module, it includes a pair of epipolar-corrected forward-looking binocular cameras and a high-frequency structured light projector located between the two cameras.

[0035] Binocular camera: Provides left and right views with parallax, serving as both a sensor for forward perception and an imaging source for constructing 3D point clouds.

[0036] Structured light projector: Projects pseudo-random speckle or stripe patterns onto the inner wall of the pipe. Key function: Artificially "textures" the smooth, textureless inner wall of the pipe, ensuring that the binocular algorithm can calculate high-precision dense depth under any pipe wall conditions, overcoming the failure problem of traditional vision on weakly textured surfaces.

[0037] High-precision IMU: Integrated within a multi-sensor module, it is used to sense the three-axis attitude and acceleration of the end effector. Acting as the front-end of data acquisition, it penetrates deep into the pipeline, performing high-frequency data acquisition driven by a hard synchronization signal.

[0038] Furthermore, the operation flow and principle of the spatial state perception gating and physical triggering subsystem are as follows: S1 Status Sentinel (Interference Detection): Principle: The FPGA motion control card reads the three-axis acceleration and three-axis angle (attitude angle) feedback from the IMU in real time.

[0039] Interference definition: The system presets a "safety threshold". For example: acceleration < 0.1g and angle deviation < 2°. Monitoring logic: Active disturbance: When the motor performs variable speed operation, the IMU will sense the change in acceleration.

[0040] Passive disturbance (human-induced): When the tester manually moves or taps the sensor module, the IMU will instantly sense a violent change in angle or a vibration peak.

[0041] Judgment: If the current acceleration or angle exceeds the preset safety threshold, the system determines that it is currently in an unsteady state (under disturbance).

[0042] S2 Step-velocity co-modulation mechanism based on parallax perception Principle: By utilizing the forward-looking perception of the front-facing binocular camera, an inverse coupling relationship between sampling density and feed rate is established.

[0043] The system first determines the vertical field of view height of a single camera in the circumferential array vision unit ( ) and preset image overlap rate ( ), calculate the maximum allowable basic step size :

[0044] Function: To ensure seamless coverage of circumferential panoramic images under standard acquisition mode, eliminating any missed areas.

[0045] Dynamic modulation logic: Parallax monitoring: The FPGA motion control card calculates the parallax change rate of the binocular ROI region in real time.

[0046] Smooth propulsion mode: When the binocular parallax d is stable, the system executes the basic step size. To advance with maximum efficiency.

[0047] Feature Fine Scanning Mode (Active Deceleration): When a parallax step is detected (predicting that it will soon cross a weld seam or change in diameter), the FPGA motion control card immediately performs two actions: Encryption Space: Switches the trigger step size to an ultra-fine step size; Active Deceleration: Forces the servo motor to decelerate through the underlying bus.

[0048] Design purpose: If only the step size is increased without deceleration at the diameter change point, the extremely high triggering frequency (f=v / s) will cause a collision with the mechanical vibration. —The IMU frequently shuts down due to excessive vibration, resulting in the loss of a large number of high-frequency trigger signals. By actively slowing down, the vibration energy during mechanical obstacle crossing is reduced, making it easier for the IMU to be released. On the other hand, sufficient exposure and transmission time windows are provided for high-density sampling, ensuring that key feature data is both dense and stable.

[0049] S3 Physical Gated Arbitration: When the motor encoder pulse count reaches the sampling step size (determined by S2), an internal "sampling request" is generated. This request is sent to the "normally closed electronic gate," and the FPGA motion control card checks the current IMU status: if the IMU value is below the threshold, it immediately triggers a photo capture. If the IMU value is above the threshold, the FPGA motion control card refuses to trigger the capture immediately.

[0050] Processing: The system enters S4 micro-delay elastic capture mode. Once the vibration wave has passed and the IMU value drops below the threshold (the moment the vibration crosses zero), a trigger signal is resent.

[0051] S4 Micro-delay Elastic Capture: Instead of skipping the current sampling point, the system actively introduces a dynamic Δt (0~50ms) micro-delay.

[0052] Strategy: The FPGA motion control card tracks the derivative of E(t) in real time to find the subsequent zero-crossing point (or energy valley value) of the vibration.

[0053] Action: Once the optimal moment is captured, immediately release the physical trigger signal.

[0054] Effect: Absolute stability of spatial attitude is achieved by utilizing microsecond-level time elasticity.

[0055] S5 Source-Level Spatiotemporal Hard Implantation: In the same clock cycle as the trigger signal is released, the FPGA motion control card extracts the current absolute physical mileage and instantaneous attitude quaternions in parallel. Using the underlying protocol, this data is combined into a 64-bit spatiotemporal status stamp and hard-written into the frame header of the sensor's raw data.

[0056] Furthermore, the operation process and principle of the truth reconstruction system are as follows: This system abandons the "soft stitching" logic based on image feature point matching or probability optimization in traditional technologies, and instead constructs a deterministic processing architecture from low-level hard embedding to high-level hard reconstruction. The host computer utilizes its CPU / GPU computing power to parse heterogeneous data streams with spatiotemporal state stamps and executes the following deterministic geometric reconstruction steps: Step 1: Reverse geometric correction based on status stamps Principle: Instead of performing image sharpness assessment, the instantaneous attitude quaternions hard-embedded in the FPGA motion control card in the data frame header are used to construct an inverse rotation matrix. This forces the perspective distortion caused by the cantilever vibration of the multi-level nested telescopic fork mechanism to be "corrected" to the ideal kinematic coordinate system.

[0057] Mathematical model: Attitude calculation: Analyzing the quaternion latched at time t in the frame header Convert it to a rotation and convert it to an instantaneous rotation matrix. .

[0058] Homography Transformation: Let K be the camera intrinsic parameter of the array vision unit. Construct a homography transformation matrix from the "current jitter pose" to the "ideal horizontal pose". :

[0059] Pixel remapping: remapping the pixels of the original image Perform correction: .

[0060] Step 2: Circular panoramic stitching based on rigid constraints Principle: The physical pulse increment of the encoder is used as the only longitudinal constraint, and rigid splicing without feature points is achieved by combining cylindrical projection.

[0061] logic: Circumferential splicing: S1 offline calibration: Pre-determine the extrinsic rotation matrix of 8 cameras It is then solidified into known system parameters (mold).

[0062] S2 cylindrical unfolding: For each frame of the original planar image Using camera intrinsic parameters Using the cylindrical projection formula, map it to a unified virtual cylindrical coordinate system. Above, edge perspective stretching caused by camera planar imaging is eliminated, generating 8 standard "curved tile images".

[0063] ,

[0064] S3 Rigid Registration: Based on external parameters calibrated offline The fixed pixel offset between adjacent cameras in cylindrical coordinates can be calculated directly. The system directly connects the right edge of the i-th image to the left edge of the (i+1)-th image. Forced overlap occurs at certain points, eliminating the need for feature point search.

[0065] S4 Linear Transition: Within a fixed overlapping area (e.g., 50 pixels), a linear weighted average algorithm is used. The weights gradually change from 1 to 0 from left to right, smoothly transitioning the pixel values ​​in the overlapping area, eliminating stitching seams caused by lighting differences, and ultimately outputting a single-frame 360° circular panoramic image.

[0066] Vertical stitching: Reading encoder pulse increments written by the motion control card integrated with the FPGA. Calculate the vertical pixel displacement between images

[0067]

[0068] ( Pulse equivalent Optical magnification (pixel size) Based on this, continuous circular panoramic images are stacked at fixed intervals to generate a distortion-free panoramic unfolded image.

[0069] Step 3: Dense 3D Reconstruction Based on Active Structured Light Principle: Abandoning the complex registration logic of heterogeneous sensors (radar + camera), the three-dimensional coordinates are directly calculated based on the parallax principle of binocular vision, and the problem of smooth pipe walls without texture is solved by using structured light.

[0070] logic: Stereo matching: The dense disparity map D(u,v) of the corrected left and right images is calculated using a semi-global matching algorithm (SGM). Thanks to the speckle texture projected by the projector, extremely high matching confidence can be obtained even on featureless smooth pipe walls.

[0071] Depth calculation: based on the principle of triangulation (where f is the focal length, B is the baseline, and d is the disparity), convert the disparity map into a depth map.

[0072] Point cloud generation: Combine the camera intrinsic parameters K to backproject the depth map into a 3D point cloud P(x,y,z).

[0073] Truth value output: Since the depth information Z and texture colors (R,G,B) both come from the same optical system, there is naturally no pixel alignment error, and the truth model with perfect geometry and texture is directly output.

[0074] Step 4: Spatial Hysteresis Alignment Based on Physical Mileage Index Addressing the inherent physical mounting distance between the front-end active binocular module and the rear-mounted array vision unit .

[0075] logic: Binocular mapping: The front-end binocular module collects and constructs a "3D geometric slice G1" at the physical mileage S1, labels it with mileage S1, and stores it in the cache queue.

[0076] Array Imaging: The array camera acquires a "panoramic texture image T2" ​​at physical mileage S2 and labels it with mileage tag S2. Hysteresis Matching: The system does not perform time alignment but monitors the encoder mileage in real time. When the array camera's mileage S2 meets the... At that time, the system determines that the texture T2 at this moment corresponds exactly to the previous geometry G1.

[0077] Rigid stitching: precisely maps texture T2 onto geometric model G1.

[0078] Specifically, the operating logic of this invention is as follows: The complete operation of this invention includes three stages: clamping and centering, extension for acquisition, and retraction and reset. Phase 1: Pipeline Alignment Pipe adjustment: Using the base limiting tool as an aid, select a suitable U-shaped support frame according to the cylindrical pipe to be tested, place the pipe to be tested on the cylindrical positioning support frame, and make the center of the pipe coincide with the axis of the telescopic fork movement.

[0079] Phase Two: Dynamic Extension Data Collection Start feeding: The servo motor drives the telescopic fork mechanism, which carries the multi-sensor module, to slowly extend into the interior from the pipe port via a command issued by the electrical control box.

[0080] Kinematic simulation engine: Integrated into the FPGA motion control card, used to generate nonlinear velocity curves that simulate the characteristics of a real robot.

[0081] Start-stop and speed change simulation: Acceleration and deceleration time constants are set to cause the motor to output step torque, simulating robot slippage, slight vibration, or impact moments. The FPGA motion control card's position comparison circuit operates independently, recognizing only position and not speed, ensuring accurate data acquisition even during the most dramatic speed changes.

[0082] Spatial Triggering: During the extension process, the FPGA motion control card inside the electrical control box reads the motor encoder pulses and binocular visual parallax data in real time. The system no longer performs blind triggering with a fixed step size, but dynamically adjusts the sampling step size according to the characteristics of the pipe wall in front perceived by the binoculars (such as the binocular parallax detection is to encrypt the sampling at the diameter change point where there may be a weld seam in front), and determines the triggering time by combining the mechanical vibration status fed back by the IMU.

[0083] Gated triggering: The FPGA motion control card releases the sampling signal via physical connection only when the position is reached and the mechanical state is stable.

[0084] Synchronous operation: The moment the multi-sensor module receives the signal, the surround-view camera exposes, the binocular and structured light project, and the IMU latches the data, completing a true value acquisition based on absolute position.

[0085] Motion simulation: According to the preset program, the telescopic fork mechanism can perform non-linear motions such as speed change and shaking to simulate the complex working conditions of a real robot.

[0086] Phase Three: Retraction and Data Transmission Reaching the endpoint: The system automatically stops when the telescopic fork mechanism reaches the preset stroke or the sensor detects the end of the pipe wall.

[0087] Rapid retraction: The telescopic fork mechanism retracts smoothly to its initial position (at the electrical control box) at high speed.

[0088] Data transmission: True data is transmitted to the host computer via a high-speed bus for storage and subsequent multi-source data fusion processing.

[0089] The above description constitutes an embodiment of the present invention. The foregoing descriptions are preferred embodiments of the present invention. Unless there is a clear contradiction or a prerequisite for a particular preferred embodiment, the preferred embodiments can be arbitrarily combined and used. The embodiments and specific parameters described are merely for clearly illustrating the verification process of the invention and are not intended to limit the scope of patent protection of the present invention. The scope of patent protection of the present invention is still determined by its claims. Similarly, any equivalent structural changes made based on the description and drawings of the present invention should also be included within the scope of protection of the present invention.

Claims

1. A full-scale simulation test system for multi-source perception of pipeline robots based on spatial sampling consistency determination, characterized in that, The system includes a mechanical execution system, a spatial state perception gating and physical triggering system, a multi-sensor sampling unit, and a truth reconstruction system. The mechanical execution system includes a cylindrical positioning support frame (1), on which a cylindrical pipe (2) is installed. A telescopic mechanism (3) is provided next to the cylindrical positioning support frame (1). The telescopic mechanism (3) includes a bottom frame (41). An electrical control box (32) is provided on the side of the bottom frame (41) near the cylindrical positioning support frame (1), and a telescopic fork assembly (33) is also provided. The central axis of the telescopic fork assembly (33) is on the same straight line as the axis of the cylindrical pipe (2). A support rod (34) is provided on the frame (31). A linear guide pair (35) for guiding the telescopic fork assembly (33) is horizontally provided on the top of the support rod (34). One end of the telescopic fork assembly (33) is sleeved in the electrical control box (32). A connector (36) is provided on the other end of the telescopic fork assembly (33). The other end of the connector (36) is locked in the linear guide pair (35) and can slide along the length direction of the linear guide pair (35). A power module and a drive assembly for driving the axial movement of the telescopic fork assembly (33) are provided in the electrical control box (31). The drive assembly includes a servo motor, a servo driver, and an FPGA motion control card. The spatial state perception gating and physical triggering system is used by the FPGA motion control card for arbitration of sampling timing, step size modulation and physical triggering; The multi-sensor sampling unit is used to execute sampling instructions and generate raw data with location tags; The truth reconstruction system is used to perform deterministic spatiotemporal alignment of two-dimensional textures and three-dimensional geometry based on physical mileage indexes, and to construct standardized pipeline truth values ​​containing position, texture and geometric information through geometric inverse correction and high-precision texture mapping. The multi-sensor sampling unit includes a multi-sensor module, which is installed on the end face of the telescopic fork assembly (33) that is sleeved on the electric control box (31); The top of the electrical control box is equipped with a human-machine interface panel. The human-machine interface panel is electrically connected to the FPGA motion control card, servo motor, power module, and multi-sensor module. The servo motor and multi-sensor module are electrically connected to the FPGA motion control card. The servo motor, FPGA motion control card, multi-sensor module, and power module are electrically connected to the power module. The human-machine interface panel is also connected to a host computer.

2. The pipeline robot multi-source perception full-scale simulation test system based on spatial sampling consistency determination according to claim 1, characterized in that, The multi-sensor module includes a housing (5), which is circumferentially arrayed with multiple high-definition cameras (6). Two forward-looking cameras (7) are arranged on the end face of the housing (5) near the cylindrical pipe (2). A structured light projector (4) is arranged between the two forward-looking cameras (7). An IMU is arranged inside the housing (5).

3. The pipeline robot multi-source perception full-scale simulation test system based on spatial sampling consistency determination according to claim 2, characterized in that, The specific operation process of the spatial state perception gating and physical triggering system is as follows: S1, Interference Monitoring: The FPGA motion control card reads the three-axis acceleration and three-axis angle feedback from the IMU in real time, and determines whether the current acceleration or angle deviation exceeds the safety threshold based on the preset safety threshold. If so, it determines that the multi-sensor sampling unit will not perform sampling. S2, step size modulation based on parallax perception: By using two forward-looking cameras, an inverse coupling relationship between sampling density and feed rate is established, based on the vertical field of view height of a single high-definition camera set in a circumferential array. and preset image overlap rate Calculate the maximum allowable basic step size. , in, This ensures seamless coverage of the circumferential panoramic image under standard acquisition mode. The FPGA motion control card calculates the parallax change rate of the two forward-looking camera ROI regions in real time. When the parallax d of the two forward-looking cameras stabilizes, the system executes the basic step size. When a parallax step is detected, the FPGA motion control card immediately switches the trigger step size to a fine step size and controls the servo motor to decelerate. S3, Physical Gated Arbitration: When the motor encoder pulse count reaches the sampling step size, a sampling request is generated. The FPGA motion control card checks the current IMU status: if the IMU value is lower than the safety threshold, it immediately triggers the photo capture; if the IMU value is higher than the safety threshold, the FPGA motion control card refuses to trigger immediately and enters the S4 micro-delay elastic capture mode; when the vibration wave passes and the IMU value falls back below the threshold, the trigger signal is resent. S4, micro-delay elastic capture: The system does not skip the current sampling point and actively introduces a dynamic Δt micro-delay. The FPGA motion control card tracks the IMU value in real time. When the IMU value falls below the threshold, a physical trigger signal is sent again. S5, a source-level hard implantation of spacetime: In the same clock cycle as the release trigger signal, the FPGA motion control card extracts the current absolute physical mileage and instantaneous attitude quaternion in parallel, and combines the absolute physical mileage and instantaneous attitude quaternion data into a 64-bit spatiotemporal status stamp, which is then hard-embedded into the frame header of the original data of the multi-sensor sampling unit.

4. A multi-source perception full-scale simulation test system for pipeline robots based on spatial sampling consistency determination as described in claim 3, characterized in that, The specific operation process of the truth reconstruction system is as follows: Step 1, Reverse geometric correction based on status stamp: By utilizing the instantaneous attitude quaternions hard-embedded in the FPGA motion control card in the data frame header, an inverse rotation matrix is ​​constructed to forcibly correct the perspective distortion caused by the vibration of the telescopic fork mechanism to the ideal kinematic coordinate system. Specifically: Parse the quaternion latched at time t in the frame header Convert it into an instantaneous rotation matrix Let K be the camera intrinsic parameter of the array vision unit, and construct the homography transformation matrix from the current jitter pose correction to the ideal horizontal pose. : ; For the original image pixels Perform correction: ; Step 2, Circular panoramic stitching based on rigid constraints: Using the physical pulse increment of the motor encoder as the sole longitudinal constraint, and combining it with cylindrical projection, rigid splicing without feature points is achieved, specifically as follows: Pre-determine the extrinsic rotation matrix of all high-definition cameras in the circumferential array. To solidify it; For each frame of the original planar image Using camera intrinsic parameters Using the cylindrical projection formula, map it to a unified virtual cylindrical coordinate system. Above, edge perspective stretching caused by camera planar imaging is eliminated, generating a standard number of curved tile images corresponding to the specified number; among them, , ; Based on the external parameters calibrated offline The fixed pixel offset between adjacent cameras in cylindrical coordinates can be calculated directly. The system will connect the right edge of the i-th image to the left edge of the (i+1)-th image. Forced overlap is applied; within a fixed overlap area, a linear weighted average algorithm is used, with the weights gradually changing from 1 to 0 from left to right to smoothly transition the pixel values ​​in the overlap area, eliminating stitching seams caused by lighting differences, and finally outputting a single-frame 360° circular panoramic image. Then, the encoder pulse increment written by the motion control card integrated with the FPGA is read. Calculate the vertical pixel displacement between images : ; in, It is the pulse equivalent; Optical magnification; Pixel size; Based on this, continuous circular panoramic images are stacked at fixed intervals to generate a distortion-free panoramic unfolded image; Step 3, Dense 3D Reconstruction Based on Active Structured Light: The 3D coordinates are calculated based on the disparity of binocular vision from two forward-looking cameras. A semi-global matching algorithm is used to calculate the dense disparity map D(u,v) of the corrected left and right images. Based on the speckle texture projected by the structured light projector, matching confidence is ensured on the featureless smooth pipe wall. The triangulation principle is then applied. Where f is the focal length, B is the baseline, and d is the disparity, the disparity map is converted into a depth map; combined with the camera intrinsic parameter K, the depth map is back-projected into a 3D point cloud P(x,y,z); since the depth information Z and texture color (R,G,B) both come from the same optical system, there is no pixel alignment error, and thus a ground truth model with perfect geometry and texture locking can be directly output. Step 4, Spatial Hysteresis Alignment Based on Physical Mileage Index: Based on the inherent physical mounting distance between the two front-view cameras and the high-definition camera array in the circumferential array. The system collects and constructs a 3D geometric slice G1 at physical mileage S1 using two front-facing cameras, labels it with mileage tag S1, and stores it in a cache queue. The system also collects a panoramic texture image T2 at physical mileage S2 using a high-definition camera array, and labels it with mileage tag S2. The system monitors the encoder mileage in real time. When the mileage S2 of the array cameras meets the specified value... At this time, it is determined that the current panoramic texture image T2 corresponds to the previous three-dimensional geometric slice G1, and the texture T2 is accurately mapped onto the geometric model G1.

5. A multi-source perception full-scale simulation test system for pipeline robots based on spatial sampling consistency determination as described in claim 1, characterized in that, The cylindrical positioning support frame (1) includes a base limiting fixture (11), and a U-shaped support frame (12) is installed on the top of the base limiting fixture (11).

Images